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ABSTRACT OF THE DISSERTATION 


An Interactive Program for 
Conversational Elicitation 


of Decision Structures 


by 


Antonio Leal 
Doctor of Philosophy in Engineering 
University of California, Los Angeles, 1976 


Professor Judea Pear], Chairman 


An interactive computer program has been designed and 
implemented that elicits a decision tree from a decision maker jn an 
English-like conversational mode. It sii pages a decision analyst who 
guides the decision maker in stniceucing and organizing his knowledge 
about a particular problem domain. The objectives of the research 
are: (1) to provide the decision analysis industry with a practical 
automated tool for eliciting decision structures where manual 
elicitation techniques are either infeasible or uneconomical, 

(2) to cast the decision analyst's behavior into a formal framework 
in order to examine the principles governing the elicitation 
procedure and gain a deeper understanding of ‘the analysis process 


itself, and (3) to provide experimental psychologists with an 


automated research foal for soane subjects'. perception of spOuien 
Situations into a standard and Forma] representation. 

The approach centers on the realization that the dynamic 
process of decision tree elicitation is almost identical to conducting 
a heuristic search on game brace Heuristic search techniques, when 
applied to tree elicitation, permit real-time rollback and sensitivity 
analysis as the tree is being formulated. Thus, it is possible to 
concentrate effort on expanding those parts of the tree which are 
crucial for the resolution of the solution plan. The program requires 
the decision maker to provide provisional values at each intermediate 
Stage in the tree construction, that estimate the promise of future 
opportunities open to him from that stage. These provisional values 
then serve a role identical to a heuristic evaluation function in 
selecting the next node (scenario) to be explored in more detail. 

The program is completely domain independent. That is, it 
assumes no prior knowledge specific to any problem environment. 
Through interaction with a prescribed sequence of queries, the 
decision maker is able to supply information on the structuring of 
his particular decision predicament. For example, the decision maker 
may supply information about the various decision alternatives 
representing possible future opportunities, outcomes of uncertain 
events along with likelihood relations among them, and immediate 
costs, if any. In addition, he is permitted to make a mental | 
"experiment" that, when actually performed at a later time, will 
sharpen the likelihood estimates of future events. The final result 
of the computer interview js a complete "solution plan” that 


‘recommends an action for all anticipated contingencies. 


I. INTRODUCTION 


Decision analysts are often called upon to assist in the 
solution of planning problems ranging over a large variety of domains. 
In such circumstances, the decision analysts usually possess less 
Specific knowledge about the problem domain than their customers and 
their contributions are confined primarily to formalization of the 
problem and optimization of the solution. While optimization is usually 
performed on electronic computers, formalization has invariably been 
accomplished manually, using lengthy interviews with persons intimately 
familiar with the problem domain. 

This dissertation describes an attempt to automate the 
formalization phase using an interactive computer system that guides 
the decision maker through a structured eng enalike dialog and 
constructs a decision tree from his responses. The objectives of this 
work are three-fold: (1) to provide the decision analysis industry [1,2] 
with a practical automated tool for setting decision trees in cases 
where manual elicitation techniques are either infeasible or uneconomical, 
(2) to cast the decision analyst's behavior into a formal framework in 
order to examine the principles governing the elicitation procedure and 
gain a deeper insight into the analysis process itself, and (3) to 
provide experimental psychologists with an automated research tool for 
encoding subjects’ perception of problem situations into a standard and 
formal representation. _ 

The absence of an established formal procedure of eliciting 
decision trees is not difficult to ipdenstand: Tree construction is the 


first step in the decision-making endeavor: the formal representation 


of informal concepts [3]. Since the informal concepts reside in the 
decision maker's perception of the problem environment, the translation 
- process Sonetees-oF discussions and snberieus as well as attempts to 
educate him as to the type of information he is to supply. It often 
requires a special insight and ingenuity on the part of the analyst to 
direct the conversation and phrase the queries in a way that will yield 
both informative and reliable information. 

From a practical viewpoint, the major drawback of manual 
interviews is their length and cost. Since real-time analysis of decision 
trees is beyond the limitation of human computational capability, it 
invariably happens that many hours of interviews are spent on eliciting. 
portions of the decision tree that do not have decisive bearing on the 
sobien at hand. This fact can only be discovered at a later stage, 
when the problem structure is formalized and when a sensitivity 
analysis has been conducted on a computer. During the interview itself, | 
however, it iS impossible for the analyst to process the entire 
information obtained by him up to that point and to select the optimum 
course for conducting his future inquires. For example, in eliciting 
utilities it is a common practice to extract indifference curves among 
several value attributes. Often the decision analyst is forced to 
elicit these curves over a wide range of stents only to find out 
later that eliciting preferences among a few selected points would 
have been sufficient to determine the entire problem [4]. Similar 
Situations occur in the process of eliciting conditional probabilities 
and a the expansion of very complex trees. Thus, our inability to 
perform real-time analysis of the information at hand forces us to waste 


precious time on inconsequential, detailed queries. =——S—=—<—S 


A direct man-machine interface could provide three distinct 
advantages. First would be the capability of real-time sensitivity 
analysis, which in turn could be used to guide the growth of the 
decision tree in only the most promising directions. Detailed queries 
could be reserved, then, for those branches of the tree which, on the 
basis of the information obtained thus far, seem most crucial to the 
main decision-related goals. 

The second advantage of direct man-machine elicitation would 
be the ease of updating the system with new knowledge. It is expensive 
(if not impractical) in many situations to solicit the assistance of 
a decision analyst each time the decision maker Gains a new piece of 
knowledge. A conversational system, on the other hand, could provide 
an intimate medium that could be updated quickly even by the non-technical 
planner. 

The third advantage would lie in the feasibility of incor- 
porating real-time Delphi methods for aggregating the opinions of 
several experts [5]. Decision structures elicited from other members 
of the team could be interrogated at will and displayed during the 
elicitation process to help an expert enrich or revise his previous 
structure. Disagreement could be detected, isolated, and brought up 
for further team discussion. | 

The art of directing a tree-eliciting dialog is governed 
by two conflicting goals. On snes haaee the snaivet attempts to bring 
to bear the most complete knowledge that ‘i decision maker may possess 
relevant to his current problem. On the pene handy ne desires to do so 


with the least number of queries. Cutting down on the number of queries 


soula ony be accomplished at the expense of reducing the reliability of 
the information obtained. The analyst could, of course, limit the a 
queries to holistic, global judgments, avoiding the painstaking 
detailed queries or he may terminate the tree prematurely, ask for 
value judgments on the terminal nodes, and begin the optimization 
phase. Such schemes, however, would defy the very purpose of decision 
analysis. 

| Decision analysis is founded on the paradigm that people 
possess reliable procedures for detecting, storing, and retrieving 
fragments of knowledge, but possess much less reliable procedures for 
aggregating these fragments into a global inference. If it were not for 
this deficiency, there would be no purpose served by analysis. It 
would then be sufficient to ask the decision maker, "Which alternative 
action seems most valuable to you?" and when he responds, advise him, 
"Do it:" The fact that the analyst refrains from asking direct value 
judgments on actions but prefers: to construct them mechanically from 
judgments on events, circumstances, and surmises, reflects the analyst's 
hope that mechanically constructed judgments, using Bayes' rollback 
‘procedures [6] are nonesarentul to the decision maker's experience 
than direct holistic - judgments: internally processed by him [7]. 
| Thus , by asking more detailed questions and expanding the depth of the 
decision tree, the analyst expects to obtain a refinement of less 
accurate judgments that could not otherwise be obtained at an earlier 
stage. It is this tradeoff between the cost of each query and its 
| contribution to the overall judgment accuracy that underlies the style 


that analysts select in conducting their dialogs. 


This tradeoff bears a striking Similarity to that which governs 
tree expansions in Artificial Intelligence procedures. Game playing, 
robot planning, and theorem proving programs [8] all seek to obtain 
close-to-optimal solutions without exhaustively searching through 
the underlying problem trees. In these applications, tree pruning is 
achieved via the use of a heuristic function: a simple rule, externally 
provided by the analyst: or programmer, that computes a crude estimate 
of the value (strength or promise) of any tree node in reaching the 
goal. In the game of chess, for example, such a rule may prescribe the 
way in which the various aspects of any board configuration (i.e. 
material advantage, mobility, number of threatened pieces, etc.) 

Should be combined to give a rough estimate of the overall strength of 
that position. The true value of each configuration (j.e. "win", 
"draw", or "lose") cannot, of course, be determined exactly unless the 
game tree is expanded exhaustively out to its terminal positions and 

a mini-max rollback procedure is applied Poa. Sine such a search is 
utterly impractical, the strength seach Sosieian is determined by its | 
rollback value after the heuristic evaluation function 1S assigned to 
the terminal nodes of a truncated tree. The strength could, of course, 
be determined by direct application of the heuristic function to the 
position in question. However, such evaluation would be far less 
reliable than one obtained by rollback over several levels of 
look-ahead. Thus, the purpose of tree Sepancien is, as in the case of 
decision trees, to obtain a more reliable estimate of a node's value. 
The value produced by the fencers Pinecone) thus be considered 

to consist of the "true" value plus an error factor or noise that is 


reduced by tree expansion. 


The availability of a heuristic evaluation function is one of 
the advantages that heuristic search has compared to decision tree 
expansion. The function is available at each node during expansion to 
estimate its relative potential for achieving the search objective. Thus, 
not only is this measure used és evaluate the tip nodes before rollback, 
but it also determines the order of node expansion. Heuristic search 
procedures select for expansion that node which, on the basis of the 
provisional evaluation function, seems most likely to separate the 
top contending plans. Using this technique, substantial savings jin 
node expansion have been achieved, even with very crude heuristic 
functions. 

In contrast, in manual decision tree elicitation, the analyst 
evar structures the entire tree (to some comfortable depth) before 
values are placed on the tip nodes. Drawing an analogy with heuristic 
search techniques, it seems that decision tree analysis could be 
enhanced considerably if values were available on nodes as the tree is 
being expanded. What would such values represent? They would be no 
different than values placed at the tip nodes of the completed tree: 
estimates of the relative utility of the outcomes. This utility 
represents a mental estimate of the rollback value of a complete 
theoretical tree emanating from that particular node on. It can, — 
as before, be regarded as the final expected value with errors due to 
deficiencies in mental aggregation procedures. Nevertheless, if the 
decision maker 1s requested to provide values for nodes as they are 


being expanded, the information can be used to determine a node 


expansion order. As the tree expands, these provisional values become 
more “refined”, that is, closer to the true (mechanically processed) 
value. 

The major objective of the elicitation program discussed 
here is the construction of the decision tree. While many other 
computer systems focus on the elicitation of utilities and proba- 
bilities [10], these aspects sire not emphasized here. The decision 
maker iS assumed to be able to provide value and likelihood estimates 
in numerical form, eliminating the need for construction from a 
sequence of binary choice queries. More sophisticated techniques for 
utility and probability elicitation could be incorporated into the 
program once the elicitation skeleton is developed. 

The first portion of this dissertation deals with the 
fundamental concepts incorporated into the elicitation program. The 
latter portion describes the operational program itself. Section II 
describes heuristic search in more detail than discussed thus far 
and relates it to the node expansion algorithm imposed by real-time 
sensitivity analysis. In section III, the opportunity to gain 
information through experimentation feeieensset and the method by 
which this opportunity is made available to the decision maker. The 
main elicitation procedure, as experienced by the decision maker 
using the program, 1S described in section IV iene with typical 
queries. Section V demonstrates the spenation of the program using 
a typical dialog with a decision maker facing a hypothetical problem 
situation. The internal structure and organization of the actual 


program is illustrated in section VI with a description of major 


routines and their relation to the entire system. Section VII 
summarizes major conclusions of the research including a discussion 
of the advantages and deficiencies discovered during the development. 


A complete program code listing is supplied in the Appendix. 


II. NODE EXPANSION ORDER 


Traditional decision tree elicitation techniques require 
the decision maker to formulate decision aeernatives and expected 
events as a first step toward tree construction. Only after the tree 
Structure has been tentatively solidified, is the decision maker 
expected to place probabilities on event outcomes and values on 
projected future situations. Probabilities are placed at each internal 
event node and values are placed at each terminal ("tip") node of the 
tree. Each tip node represents a projected future <aeustion (scenario) 
that is uniquely determined by the path from the initial decision 
(root) node to the tip of the tree. The decision maker must evaluate 
each individual path and place a numeric value on it in terms of a 
previously selected utility scale. After all values and probabilities 
have been determined, a computational (rollback) analysis determines 
which of the initial decision alternatives is the most favorable for 
the decision maker to take. 

The current research emulates an "automated decision 
annie who, aside from recommending optimal action plans, also 
constructs the decision tree itself. It calls for a rational 
procedure to govern the growth of the decision tree. The adopted 
approach to such a procedure rests on the hypothesis that expanding 
the decision tree during an elicitation interview is equivalent to 
conducting a heuristic search in a complex problem Sica. As an 


example of problem solution by search, consider the task of finding 


the shortest route between ae CARIES: say Los Angeles ane Mauston. 
In this domain, the search "tree" is a representation of possible 
routes from Los Angeles to Houston and a "node" represents a highway 
junction. A search procedure would investigate each route, one at a 
time, in an attempt to locate the optimal path. It is, essentially, 
problem solution by exhaustive enumeration. The final result is an 
optimal solution path that recommends a course of action at each 
node. In this case, the result would be a list of "directions" which 
Specify a highway to follow at each junction. 

The game of chess provides another example of a problem 
which can be attacked by search techniques. In this case, however, 
it 7s not a solution path that is required, but a solution plan. 
Each node in the search tree represents a different chess board 
configuration. The nodes are connected by the legal moves of the 
game. That is, the transition from one node (board position) to 
another in Piascenten tree is governed by the possible chess moves 
that the player can make at that time. Since the moves of the opponent 
are unknown, a solution plan must recommend an optimal move regardless — 
of his actions. Such a plan is an optimal "subtree" contained in the 
original tree of all possible moves. 

‘The search procedure itself begins at the initial situation, 
called the "root" node, and generates possible successor nodes 
according to the characteristics of the specific problem domain. 
In the cities example, this corresponds to locating each highway 


junction leading out from Los Angeles. In chess, each possible move 


from the current position is considered. Once this generation is 
accomplished, the node is said to have been "expanded". By repeated 
expansion, longer and longer paths are generated and the search tree 
is "grown". | 

Aithough the expansion rules follow straight-forwardly from 
the problem environment, the order of expansion remains to be defined. 
That is, there are no dacensic rules that select the next node to be 
expanded. Two well-known tree search procedures are available that 
search the nodes in a uniform or "blind" manner. They are "breadth- 
first" and “depth-first” expansion [8]. Uniform node expansion 
algorithms generate an expansion order that is independent of any 
information about the relationship between the candidate nodes and 
the solution. Breadth-first expansion grows the tree on equal-depth 
contours (see Figure 2.1). All nodes in the outer-most layer are 
expanded before any node in deeper layers. This insures that al] 
paths from the root have equal length. In be coneene of decision 
tree elicitation, for instance, a breadth-first expansion order means 
that each scenario has an equal level of refinement. Thus, tree 
growth will, in general, follow an ares wrogieesion in time and 
the decision maker will not be forced to make predictions ina 
widely varying time domain. The disadvantage is, of Baueee. that he 
is forced to elaborate non-crucial scenarios. 

The second uniform expansion algorithm 1s "depth-first" 
(see Figure 2.2). Each path is explored to its logical conclusion 
(or to a prespecified depth bound) and henqnetaeed paths are 


generated in an overall left-to-right pattern. In the context of 


ae 


Figure 2.1. Breadth-First Expansion Order 


Figure 7d ae Depth-First Expansion Order 


decision analysis, the decision maker would be accea to follow a 
scenario to its conclusion, however, precious time could, again, 
be wasted in exploring scenarios that have low impaet on discovering 
the solution. 

| In contrast to the uniform expansion algorithms described 
above, heuristic search techniques attempt to use information about 
the relationship between candidate nodes and the solution. Such 
information is in the form of a "heuristic function" which is a rule 
for estimating the difficulty of reaching the solution from any 
given node. In the cities example, a heuristic may be the direct air 
distance from any highway junction to Houston (if known). In chess, 
a heuristic would estimate the difficulty of achieving a checkmate. 
from any given game configuration. The numeric value of the heuristic 
has the effect of ordering the candidate nodes as to their promise 
in finding a solution. That node with the most promise can be 
selected as the next to expand. 

A "perfect heuristic"* should produce a value of | for all 
nodes in the optimal solution plan and a value of O for all other 
nodes. In this case, there would be no "search" since a perfect 
strategy would exist for accomplishing the objective. In real-world 
situations, however, we must usually be satisfied with a less-than- 
perfect heuristic function and a finite-horizon search tree. This 
¥ A "perfect heuristic’ is somewhat of a contradiction in terms, since 
the connotation of the word "heuristic" is a "rule of thumb" that 


comes from human intuition. 


ela 


will cause some nodes to be expanded that do not lie in the optimal 
solution plan because the heuristic function will provide a non- 
optimal ordering of the candidate tip nodes. | 

Figure 2.3 compares heuristic search with decision tree 
expansion. The method used to direct the interview requires the 
decision maker to estimate euonterondl values that represent oppor- 
tunities open to him by anticipated action/event combinations. 

The provisional values are used in the same way as the 
heuristic function in tree searching. The difference is that the 
heuristic function is initially supplied by the analyst or programmer 
and must be defined over the entire problem domain, while the 
provisional values are supplied by the decision maker during the 
elicitation interview. As a consequence, node expansions follow the 
decision maker's perception of event relationships in the environment 
rather than following predefined transition rules. These values can, 
as in the case of heuristic search, be considered to be the "true" 
values with error due to "noise". The amount of error is dependent 
on the size of the tree and the decision maker's ability to provide 
correct information. Since the decision maker's predictive ability 
remains relatively constant, the amount of error is more directly 
related to the size of the tree. | | 

The order of expansion algorithm Should be consistent with 
the overall objective of the decision ra oe Thus , the primary 
interest is not, as might be expected, to expand the most valuable 


(highest utility) path discovered, but rather, to expand that node 
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HEURISTIC SEARCH ON GAME TREES 


Es to move 


s 
be 
a 
a . [_] First player's turn C1 Decision node 


C) Second player's turn C) Event node 


to move 


-e@ Object is to find the path (olan with the 


highest heuristic value H(S) with the 
minimum number of node expansions. 


6 Complete tree unavailable explicitly. 


(Impl icitly contained in game rules.) 


@ Expansion follows state transition rules. 


(Legal moves. ) 


Heuristic function provided by analyst. 


e Heuristic function guides search. 


e Mini-max rollback. 


e Terminal nodes determined by rules(win/loss) 


DECISION TREE ELICITATION 


¢ Object is to find the path (plan) with the 
highest utility U(S) with the minimum 
number of questions. 


e Complete tree unknown to the analyst. 
(Resides in the decision maker's knowledge. E- 


e Expansion follows the decision maker's _ 
perception of event/action relationships. 


@ Provisional values provided by decision maker. 
e Provisional values determine next question. 


e Expecti-max rollback. 


@ Terminal nodes determined by decision maker. 


Figure 2.3. Analog between Heuristic Search on Game Trees and Decision Tree Elicitation 


which is most ltkely to change the currently best initial decision. 
If this particular node is not part of the subtree leading outward 
from the currently highest initial branch, then its value must be 
tneremented enough to bring the value of the initial branch leading 
to that node up until it equals the value of the highest branch. 
Thus, the sensitivity measure consists of estimating the amount of 
change (differential) in a given provisional node value necessary 
to cause a change in the currently best initial decision. 

For example, in Figure 2.4, a partial tree is shown with 
initial decision branches by, bos and bz. Branch by is shown with an 
expanded event node that has two outcomes A and B. Assume that from 
a previous rollback calculation, the values of the three decision 
branches are 5, 3, and 2 respectively. Thus, by represents the | 
currently most promising decision. To calculate the sensitivity 
differential of node A, the following question is posed, "How much 
should the vane of node A (currently 5) be raised so that the value 
of the initial branch leading to node A (i.e. b>) will equal the 
currently highest branch (i.e. b,)?" Branch bs must obviously be 
‘incremented by at least 2, but node A Sonicare iene 20% 
of its value to it. Node A must be raised £6 a total higher than 
1S in order to cause b> to exceed b,. Thus, the sensitivity dif- 
ferential of node A is 10. Similarly, B must be raised to 5 
(assuming no other changes) in order to ise b> to increase by 2. 
Since A must be raised more than B, the value of A is said to be 


more "robust" than that of B. The formula for the sensitivity 


Figure 2.4. Basic Sensitivity Differential 
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differential is straight-forward. It can be shown for any node n, that 
if the path leading to n contains only event nodes, the sensitivity 


differential S(n) is: 


where Ab, is the dv ffevence betueen the currently highest initial 
decision branch b,., and that decision branch b, which leads to node n, 
and mP, is the product of all the probabilities along the path from b, 
to n. Nodes in the subtree leading out from b,., itself are simply 
compared to the second highest initial branch value to see how much 
they would affect b,., if they were Zowered. A somewhat more elaborate © 
computational procedure (ese ped below) is required to calculate S(n) 
across a path containing both decision and event nodes. 

The proof of equation (1) is as follows. Consider, tem- 
porarily, a decision tree consisting ‘solely. of event nodes except 
the root which is a decision node. Figure 2.5 is an example of such 
a tree. Let d be the root decision node and let n be an arbitrary 
tip node with a provisional value of Vin}. Let en be that event node 
which is a successor to d and leads to the subtree containing n. 
Let e,., be the event node successor to d with the maximum rollback 
value. Every non-initial branch in the tree must have an associated 
probability ofoccurrence. Let Pre nan = P(e,),..sP(n) be the 


sequence of probabilities from node e 


, to node n along the entire 


path. 


Ao 


-V(d) =v 


Cmax 


) 


Gna 


Figure 2.5. An Event Tree 
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Now, if the subtree following e, is collapsed to a single 
level, the number of branches will equal the number of tip nodes in 
the original subtree. That is, every path in the original subtree 
will be represented by a single branch in the collapsed tree. If 
the probabilities of the branches are assigned to reflect the product 
mP(e,,n) for each tip node, as shown in Figure 2.6, the expected 
value V(e,) will not be altered. The value V(e,) may be expressed 


as follows: 
(2) V(e,) = mP(e,,n)V(n) +k 


where k is the expected value of all branches extending outward from 
e, except n. Equation (2) simply represents the expected value of 
node e, with the single term for node n written separately and 

the pooeraneseed aes constant k. Now let V'(n) be a new value for 
node n such that V'(n) = Vin), and such that as a result of assigning | 


V'(n) to n, the value of e, is raised to equal Vie Thus, 


max) 


(3) v(e nP(e, n)V'(n) + k 


max? 


The sensitivity differential of node n is defined to be the dif- 
ference between V'(n) and V(n). From equation (2) and (3), the 


difference iS given by: 


Ne 


-_— : 
2. 
ee 


OKE 


ee 


Figure 2.6. Collapsed Event Tree 
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V(enay)-k V(e,)-k 


(4) V'(n)-V(N) = mP(e,sn)  P(ep,n) 


Consequently, from equation (1) we have: 


Vlemay) -V len) 


mP(e, nh) 


(5) s(n) = 


For notational convenience, this may be written: 


(6) s(n) = ——= 


It now remains to show how decision nodes between ey, and 
n affect the sensitivity differential measure. Figure 2.7 shows a 
decision tree partitioned into mutually exclusive subtrees such 
that sock subtree has a decision node at its root and contains 
nothing but event nodes. The sensitivity differential of each tip. 
node may be calculated relative to its own root decision node 
inside each partitioned subtree. To connect the subtrees, simply 
consider the value of each subtree as the me of a tip node for 
the immediately preceeding subtree. That is, every decision node 
Peucent the initial one) is seen both as the root of its own subtree 


and as a tip node of the predecessor subtree. | 
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Figure 2.7. Partitioned Decision Tree 
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In Figure 2.7, node d, is the initial decision node of the 
entire tree and, of course, has as its value the maximum value of 


its successors. That is, 


(7) Vids) = Vice.) 

The node do plays two roles. It is the initial decision node of 

its own subtree and a tip node to the dj subtree. Consider node m 
located in the d, subtree with a provisional value of V(m). As | 
shown before in equation (5), it is possible to calculate both 

the sensitivity differential of node m with respect to do (denoted 
S(m/da ) ) and the sensitivity differential of node do with respect to 
dy (denoted S(do/d, ) ). For reference, these are: 


V(e'max)-V(em) 


(8) S(m/do) = sP(dp,m) 


V(emax)-Vleg,) 


(9)  S(do/dy) Ha 


The task now is to calculate S(m/dy): the sensitivity differential 
of node m with respect to d,. 
This is done in a two-step process for the example in 


Figure 2.7, beginning at d, and working forward toward m. Thus, 
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sensitivity calculations take place in a forward direction (from the 
root to the tips) as opposed to rollback calculations which take place 
in the reverse direction (from the tips to the root). The sensitivity 
differential S(d./d,) is the increment on V(d5) necessary to bring 


The new, higher value V' (do) 


Vleq,) up equal to the value of VAC): 


needed for this is simply the old value plus the differential. 
(10) Vi(do) = W(dp) + S(do/d,) 


Now, the amount that v(m) needs to be raised to bring V(do) up to 


V'(do) is given by: 


V'(d5)-V(e,,) 


(11) S(m/V' (do )) Poa 
; T os 


That is, in computing the sensitivity differential of node m, 


rather than raising V(e,,) up to V(e' , it must be raised further 


| max) 
to V'(d5). The value V'(d,) will always be greater than or equal to — 


that of Vier ass) since: 


(12) V'(do) = V(do) = V(e'na) 


This two-step process leads to a generalized recursive 


procedure for finding the sensitivity differential of any node in 


set 


the tree with respect to the initial decision node: 


S(n) | 
Piny ach an event node n 


S(n)+V(n)-V(T(n)) for a decision node n 


where r(n) is the successor of node n and P(n) is the probability 
along the branch from n to I(n). | 

At first glance, the node with the lowest sensitivity 
differential may appear to be crucial and should be chosen as the 
next to expand. It may be argued, however, that the factor which 
determines the node to be selected for expansion is not the absolute 


sensitivity differential S(n), but the relative sensitivity Sp(n}: 


(14) S.(n) = 


where o, (n) is the anticipated variation in the provisional value 
of node n which is likely to take place by further refinements. 
o,,(n) represents, therefore, the magnitude of the error present in 
the provisional value estimate V(n) and S,.(n) represents the 


likelihood that this error would result in a change of plan. The 


error o,,(n) may depend on the magnitude of the value. For example, 
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a node with a value of 10 is less likely to be raised to 15 by 
refinement than a node with a value of 110 is to be raised to 115. 
Equation (14) is based on the following noise model. Let 
v be the provisional value of some tip node as reported during the 
elicitation interview. Let v* be the "true" value that would result 
if this node could be theoretically expanded completely. The difference 
Av = | v¥-v{ is the error due to "noise". Figure 2.8 shows these 
quantities in the form of a probability distribution graph. The 
value v* is not known. However, treating v* as a random variable, 
we wish to find the probability that expanding a node with value v 
will cause a change in the initial decision. Let v, be the required 
value for a decision change, as calculated by sensitivity analysis. | 
Then, we wish to find P(v*>v] v). To find this probability, some 
distribution P(v*|v) must be assumed. We assume here that P(v*>v, |v) 


1S some monotonic increasing function of: 


(15) = 


where oC is the variance of v*, which leads to equation (14) as a 
proper basis for node expansion. 

| The value o,(n) cannot, of course, be computed exactly. 
It can, however, be estimated either by asking the decision maker 
directly to assess the reliability of his value judgment V(n), in 
the. form of a utility interval, or by assuming a reasonable reli- 


ability model in the form of a functional relationship oy(n) = F(V(n)) 


ale 


P(v*| v) 
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Figure 2.8. Distribution Graph 
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connecting o(n) to the magnitude of the value estimate V(n). The 
distribution spread may depend on the we ors in a manner shown 
in Figure 2.9. In the current program version, a linear noise model 
iS assumed: o,,(n) = atbV(n) reflecting the fact that ppenier'< 
inaccuracies are anticipated in assessing scenarios involving 
higher stakes. By comparing the provisional value V(n) with its 
rollback value over many nodes, it is possible to collect statistics 
on the factors which determine the reliability of human assessments. 
These could be incorporated to construct more refined models of 
value reliability for use in subsequent runs. 

Once o,(n) is determined, the value of the relative 
sensitivity S.(n) can be computed for all the tip nodes of the 
partial tree under analysis. The one with the lowest value would 
be selected for expansion. Needless to say, such analysis cannot be 
performed during a manual interview, as it involves real-time 


manipulation of the entire tree at hand. 


Figure 2.9. 
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IIT]. GAINING INFORMATION THROUGH EXPERIMENTATION 


Among the more difficult estimates for the decision maker 
to formiiate are the probabilities associated with outcomes of 
events [11]. Any chance to express and incorporate underlying causes 
or related events in order to obtain more accurate probability 
estimates should be exploited. The option to perform an "experiment" 
that will yield information about the probabilities of a related 
event provides such a chance. The option may be thought of as "buying 
information" since there is usually an experimentation cost and part 
of the analysis is to determine if the information is worth the cost.[12]. 
The experiment may be performed by either an actual physical act (i.e. 
call your stock broker) or by an internal act of recalling pertinent 
information (i.e. analyze clues that lead to a certain belief). 

Figure 3.1 shows the structure of an experiment. It takes the 
form of a two-branch decision node followed, on one of the branches, by 
a single event node. The decision node represents the choice of 
performing or not performing the experiment. If the experiment is to 
be performed, the event node represents the possible outcomes or 
"observations". Each observation has an associated probability of 
| occurrence and the experimentation cost, if there is one, must be 
incurred before the outcome is known. 

Before a successful roliback procedure can take place, all] 


probabilities relevant to the experiment must be determined. They are: 
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Possible 
Experiment 


— Outcomes 


cost 


no experiment 


Figure 3.1. Experimentation Structure — 
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the apriori event probabilities for the 


| 
uu 
~ 
oO 
ed 


jl 

; outcomes C1,--5C,, 

A P(E.) the probabilities of the experimental 

1=] | 

observations E,,...,E., and 

n m " 

A A P(C5/E.) the observation-conditional probabilities 
ie) jel 


of the event outcomes. 


The symbol A 1s used above as a short-hand notation with the 


following definition: 


m 
The apriori event probabilities A P(C.) are assumed to be known. 


However, the probabilities of the experimental observations 3 P(E.) 
are generally not known. The traditional method of ieee an is by 
first determining the event-conditional probabilities A A P(E.| C5) 
which are the reverse of the saat Ladies banih anita hebilas 
mentioned above. The P(E.) are then calculated using probability 


Summing: 


ne n m | 
A P(Es) = A = P(E;| C3)P(C,) 


and the observation-conditional probabilities are obtained by using 
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Bayes’ rule: 


PCC. Es) 
A Pee) = 
1 j=] Jo] 


m 7 P(E; |C5)P(C3) 
A ———— 
1 jel P(E.) 


los 
i>s 
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As an example of this approach, consider the physician who 
Suspects, on the basis of previously observed clinical Signs, that a 
patient has a specific disease. In order to refine his probability 
estimation, further tests can be performed that will yield more 
accurate information about the patient's condition. However, the tests 
may not be entirely specific. to the suspected disease. 

The physician may have difficulties estimating the observation= 
conditional probabilities required, that is, the probabil ity that the 
patient has the disease given that the test eesults were positive, etc. 
The reason is that his knowledge may not be organized in this fashion. 
It is more likely to be organized sound cause-and-effect relationships. 
It is the disease that causes the positive test results and not the 
other way around. Thus, the physician is more likely to be able to 
estimate the event-conditional probabilities: the probability that 
the test results will be positive given that the patient has the 
disease, etc. Because of this human tendency to organize knowledge 
around cause-and-effect relationships, it is more common to elicit 
the required conditional probabilities in a "reverse" direction, | 
that is, the probability that a particular experiment outcome is 


observed given that the event “is about to" ‘occur. 
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If, in fact, the observation-conditional probabilities are 
easier to elicit than the event-conditional probabilities (as would be 
the case in purely clinical situations where no disease model exists), 
then the path probabilities P(E. ) and P(C.| E) can be obtained directly 
without resorting to Bayes' inversion formula, assuming, of course, 
that in such situations, the observation probabilities were also 
directly estimatible. 

During the elicitation process, the opportunity for an 

experiment will normally be discovered at the time probabilities 
are being estimated for the outcomes to some particular event node. 
Thus , the experimentation structure must be inserted into a previously 
generated portion of the decision tree. Figure 3.2 shows the placement 
of the experiment (indicated by a diamond-shaped node) relative to the 
affected event node. The experiment must, of course, appear before the 
event node in the tree but need not be immediately adjacent. It must 
appear somewhere along the path from the initial root node to the 
affected event node. this is because the experiment, after all, should 
be used by: the decision maker to change his actions in accordance with 
its outcomes. There is thus a completely new set of probabilities on the 
event node for each observational outcome of the experiment. These 
sets are required . in addition to the apriori set of probabilities 
given by the decision maker before the requirement for an experiment 
arose. 

| Figure 3.3 shows the traditional representation for an 
imbedded experiment. Along one of the tree paths extending outward 


from the primary decision node, the experimentation structure is 
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Figure 3.2. Imbedded Experiment 
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Figure 3.3. Duplication of Tree Structure 
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encountered. There are n+l branches extending outward from the 
structure viewed as Sele. One branch, E,, leads to the subtree 
containing the event node with apriori probabilities P(C;),...,P(C,) 
for m branches , where P(C.) is the apriori probability that event | 
outcome C. will occur. The other n branches of the experiment lead to 
n duplicate subtrees, each identical except for the probabilities on 
the event node. In place of ee previously known apriori probabilities, 
there are mn conditional probabilities P(C,] Ej) for n observational 
experiment results i=1,...,n and m event outcomes j=1,... 5m. 
Consider, now, the problem of tree duplication as described 
above (Figure 3.3). In a static environment, subtree duplication 
caused by the insertion of an experiment Structure may not be a 
hindrance to successful decision analysis. However, in the dynamic 
environment of interactive tree elicitation, intolerable reference 
problems arise. Every time a new node is expanded, duplicates must 
be added to other parts of the tree that aye resulted from previous 
repetitions. Even if a graph structure is allowed, duplication would 
still occur in the area between the experiment and the affected event 
node making reference to specific internal nodes.ditticule: | 
What is required is a technique that eliminates the neces- 
sity for subtree duplication due to an experiment. The solution found 
to this problem is to consider the entire experimentation structure 
as consisting only of a single node with eee branch. Since the 
stnictare 46 always the same (a two-branch decision node followed by 


a single event node as shown in Figure 3.1), as long as all relevant 


ss 


information about the experiment is retained, the problem of subtree 
duplication disappears. 

Figure 3.4 depicts this transformation of the experimentation 
Structure into a single, diamond-shaped node. The cost is placed on the 
Single branch along with the n observation probabilities stored as a 
vector. The "no experiment" decision branch shown explicitly in the 
full structure is not shown as part of the experiment node but is, 
nevertheless, considered to be implicitly part of it. 

It is now possible to superimpose all of the duplicated 
subtrees and form a single unique subtree extending outward from the 
Single experiment node branch. Since the only existing difference in 
all of the duplicated subtrees is with the conditional probabilities 
at the affected event node, a single collapsed event node can serve 
to represent all of the duplicated ones. Figure 3.5 shows the method 
of storing the conditional probabilities on the peanchese Each branch 
holds a vector of probabilities corresponding to the probability of 
the particular event outcome given each of the experimental obser- 
vations respectively. Since each branch contains as many probabilities 
as experiment outcomes, there are the same number of probabilities 
in each outcome vector as there are in the single experiment vector 
(see Figure BA): It should be obvious that this representation is. 
isomorphic to the traditional duplicative representation, but forms 
a much “cleaner” tree. 

| Having a node in a decision tree with vectors of probabilities 
on each branch certainly demands a change in the expected-value 


rollback formulas. The primary objective in the definition of new 
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Figure 3.5. Collapsed Event Node 
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sommes will be, of course, i peesare the 4somorphism of repre- 

| sdntetion: That is, the internal values of all of the nodes should 

be the same regardless of which representation is chosen. Figure 3.6 
shows the computation method at the event node containing vectors of © 
observation-conditional mechanics on the branches. Each set of 
probabilities P(C.| E,) is taken, one at a time for ae 
multiplied by the tip values  V 


at 
together. The result is a vector of values corresponding to the n 


rr Ven respectively, and added 


~ values that would have resulted if each event node in the duplicated 
subtrees had been calculated independently. 
The rollback formula that accepts vectors of probabilities 

on event nodes is not the only one that must be defined. The predecessor 
to the node described above will have a vector as one of its tip values. 
Since this node may be either a decision or an event, two extended 
rol |back formulas must be defined to accept vectors of values at the 
tips. Figure 3.7 ‘and Figure 3.8° show these two formulas. It is assumed 
that only one of the branches has a vector and that the others have 
natural single values. The rollback procedure at the decision node 
takes the maximum of the combination of all of the single values 
with each of the vector values respectively (Figure 3.7). The result 
. 1S again, a vector of values. Similarly, the event node ‘calculates n 
expected values using each of the n vector values once, with a vector 
of n values as the result (Figure 3.8). | 

| Each tine the value of a node is calculated, the vector is 
propagated back toward the root of the tree until the experiment node 


is reached. Since this node caused the duplication in the first place, 
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Figure 3.6. Event Node Rollback Function 
with Probability Vectors _ 


44 


times 


Figure 3.7. Decision Node Rollback Function 
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Figure 3.8. Event Node Rol back Function 


with a Value Vector 
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the value of the experiment ~ is ie number. Figure 3.9 shows 
how the vector is collapsed. Keeping in mind the "internal structure” 
oo an Syeerinene node (see Figure 3.1), the vector of probabilities 
on the single branch must have one less member than the vector of values 
at the tip. This is due to the fact that the experiment structure has 
one exit branch with no probability attached to it: the "no experiment® — 
option on the decision node. It is assumed, by convention, that the 
value corresponding to this branch is always contained in the first 
position of the vector, labelled V(E)). Thus, it is the task of the 
rollback function for experiment nodes to take the expected value of 
the n vector elements i=1,...5n with the n probabilities, respectively, | 
and subsequently compute the maximum of this single value and V(E,)). 
It should be clear that if the above procedure is followed, 
the isomorphism between the two representations will be preserved and 
‘the ultimate value of the initial decision node will be the same in 
either case. The only remaining question is, "What happens when two 
experiments interact?" It is possible that between one experiment and 
its corresponding affected event node, another experiment could exist 
that affects a second event node further down the road. Figure 3.10 
shows an example of this seuabion: Node B is an experiment that helps 
to estimate the probabilities at node G. (shown by the dotted line). 
Node E is an experiment affecting the probabilities at node H. As 
might be imagined, the two interacting experiments form a matrix 
of conditional probabilities at node G. In order to clearly represent 
this situation, Figure 3.11 eveands both of the experiment structures 


in Figure 3.10 and shows all duplications assuming two observations 
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Figure 3.9. Rollback Function for Experiment Nodes 
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Figure 3.11 Expanded Experiment Structure 
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for each experiment (three branches in all). The two experiments 
produce o duplications of node G and its successors. 

Assume that eccrine B has observations By and By with a 
"no experiment" option of B,. Also, let E,, E,, and E> stand for these | 
Same quantities for the second experiment: node E. Then, the matrix of 


observation-conditional probabilities would appear as follows. 


The apriori probabilities would be found in the (By,E,) entry of the 
matrix. The other entries would contain all possible combinations of 


observation-conditional probabilities: 


for each probability P(G;) extending outward from node G. 


BT 


Notice the difference in the tree representations in Figure 
3.10 and Figure 3.11. Collapsing the experimentation structure into 
a single node provides a great many advantages with a small burden on 
computational effort. The tree is cleaner and can be described more 
compactly with less confusion; events, which are naturally thought of 
as Single entities, are preserved as such; and interactive decision 
analysis. is aided by having a storage representation that is closer 


to the human conceptual model. 


IV. THE ELICITATION PROCEDURE 


The sri eitericn procedure is the cycle of logical steps 
involved in obtaining detailed information from the decision maker 
about one particular node in the decision tree. The information, 
which includes provisional values, estimated probabilities, etc., must 
be sufficient to successfully perform rollback and sensitivity 
calculations. The information is obtained through a prescribed 
sequence of interactions and when completed, the node under analysis 
is said to have been “expanded”. 

Figure 4.1 shows an overview of the major steps comprising 
the elicitation procedure and their effect on the expanded node. The 
first step of the cycle consists of selecting a node for expansion 
from those available tip nodes. As will be explained later, not all 
tip nodes are available for further exploration and refinement. The 
decision maker may have indicated that in certain areas of the tree, 
enough detail has been established and no further analysis is necessary. 
Such nodes are called "terminal". The node selection is done through 
the process of sensitivity analysis described in section II. After 
node selection 1S accomplished, the node type must be detemined: The 
- type is assed as "decision" or "event" and a list of decision 
alternatives or event outcomes, as the case may be, is requested. 
After determining that the members of this "successor" list are 
mutually excluSive, an iterative procedure is initiated that elicits 


a provisional value, a probability (if necessary), and a cost for each 
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_ SELECT NEXT NODE FOR EXPANSION 
. DETERMINE NODE TYPE 


. ELICIT ALTERNATIVES OR OUTCOMES —— 


. DETERMINE IF MUTUALLY EXCLUSIVE © 


. ELICIT PROVISIONAL VALUES 


. ELICIT PROVISIONAL PROBABILITIES 


ELIE costs 


. REQUEST FOR EXPERIMENTATION | 


Figure 4.1. Elicitation Procedure SS 


of the members of the list. Then, a request for an experiment is made 
to ascertain the decision maker's confidence about his estimated 
probabilities. This finishes the cycle and, after rollback and 
sensitivity calculations, a new cycle can begin. Each of the above 
steps will now be described in detail. 

Upon the selection of a new tip node to be expanded, the 
decision maker must be alerted that he is to shift his attention 
to a specific area of discourse (i.e. a node in the tree). If the node 


is the successor of a previous decision, typical alert messages would be: 


SUPPOSE THAT YOU HAD CHOSEN TO xX 
ASSUMING THAT "X" WAS PICKED 
SUPPOSE THAT "X" IS YOUR CHOICE 
WHAT IF YOU CHOOSE TO X 

LET US SAY THAT YOU TOOK "Xx". 


The "X" referenced above is the exact name given by the decision 
maker for the particular decision alternative in question. The 
alert messages do not form a complete English sentence because they 
are the precursor to the elicitation request for the node type (see 
below). If the preselected node is an event rather than a decision, 


Slightly modified alert messages are more appropriate and desirable: 


SUPPOSE THAT "X" HAPPENED 
“WHAT IF "X" OCCURS 
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ASSUME THAT "X" HAS HAPPENED | 
LET US SAY THAT IT WAS "X" THAT ACTUALLY TOOK PLACE 


where "X" now stands for the event name itself. The alert messages 
not only serve to orient the decision maker to a particular area of 
discourse, but also serve as a prelude for interrogation of node type. 
The messages in the above two sets (as with all message sets following) 
are phrased to say the same thing in different ways to reduce monotony. 

Once the decision maker has been informed as to the sea of 
exploration, the node "type" must be determined. The type will form the 
basis for classifying the node as a major decision with alternatives 
or an event with outcomes. The interactive program allows 5 possible 
node types: | 

(1) Unknown | 

(2) Terminal 

(3) Decision 

(4) Event 


(5) Experiment 


Before expansion, the node is assigned type "unknown". At expansion 
time, the node is resolved as being of type "decision", "event", or 
"terminal". If the node is determined to be of type "terminal", it is 
assumed that the decision maker is no longer interested ‘i pursuing 
this particular scenario to any greater level of detail. It 1s 


immediately abandoned and its current provisional value (obtained at 
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a previous time) is considered final. It is removed from the list of 
available tip nodes for expansion. If the node is not terminal, the 
decision maker must declare it either decision or event. Finally, nodes 
of type "experiment" need not be considered here because they are always 
inserted into previously constructed portions of the tree and never arise 
as a result of natural node expansion. A detailed discussion of the 
elicitation of experiment nodes is presented later in this section. 

| The following queries, which are typical of those used to 
determine a node of type "decision", are meant to complete the sentence 


started by the alert messages given above: 


IS THERE A DECISION TO BE MADE AT THIS POINT? 
WOULD THERE BE OPPORTUNITIES OPEN TO YOU NOW? 
ARE SOME OPTIONS AVAILABLE TO YOU? 

DO YOU HAVE A CHOICE OF ALTERNATIVES? | 


If a negative response is received from the above, the following 


questions probe for "event" nodes: 


ARE THERE SOME EVENTS THAT MAY HAPPEN? 
ARE EVENTS ABOUT TO HAPPEN OVER WHICH YOU HAVE NO CONTROL? 
THEN PERHAPS. UNCONTROLLABLE OUTCOMES MAY OCCUR? 
CAN YOU THINK OF THINGS THAT MAY HAPPEN AS A RESULT 
OF THE CURRENT SITUATION? 


oF 


of course, the above questions are asked with the assumption 
that the responses will have some influence on the decision maker's 
future courses of action or will be relevant to his problem. If the 
decision maker fails to give a positive response for Si aneeiSi on or 
event node types, a terminal node situation is suspected and a confir- 
mation is initiated. It is assumed that enough refinement has occurred 
on the current scenario and that the decision nae is ready to explore 


other areas in more depth. Typical confirmation queries are: 


DO YOU WISH TO STOP EXPLORING FURTHER IN THIS DIRECTION? 
HAS THERE BEEN ENOUGH DETAIL EXPRESSED SO FAR? 

SHOULD WE EXPLORE SOME OTHER POSSIBILITIES IN Oe AREA? 
PERHAPS WE SHOULD TALK ABOUT SOMETHING ELSES: 

I ASSUME THAT WE CAN LEAVE THIS SITUATION? 


If a negative response is received to the siavencueet one: it is 
assumed that some misunderstanding has taken place and a short 
tutorial is started with the intention of ascertaining which of the 
three node types is preferable and forcing a selection among them. 

If the decision maker indicates two terminal nodes in succession, he 
is asked if he wishes to end the interview. 

With the node type determined, elicitation of decision | 
alternatives or event outcomes can commence. The decision maker is 
simply asked to list them using short GESerIPUIVE phrases. There is no 
limit to the jength of the list but it must ‘contain at least two 


alternatives. Typical request messages for decision alternatives are: 


Bes” 


PLEASE LIST THE ALTERNATIVES THAT YOU HAVE, ONE AT A TIME. 
STATE THE CHOICES THAT YOU HAVE. 

LIST THE OPTIONS THAT ARE AVAILABLE. 

PLEASE LIST THE DECISIONS THAT YOU COULD MAKE. 

TELL ME WHAT IS AVAILABLE. 

WHAT OPPORTUNITIES ARE THERE? 


A simple keyword search on the response determines when the list is 


complete. Typical requests for outcomes to events are worded as 


follows: 


PLEASE LIST THE OUTCOMES. 
EXACTLY WHAT EVENTS COULD OCCUR? 

STATE THOSE EVENTS THAT MAY HAPPEN. 

LIST THE POSSIBLE OCCURRENCES THAT YOU FORESEE. 
WHAT DO YOU SUPPOSE COULD HAPPEN? | 


With each response of a decision alternative or an event 


outcome as the case may be, a new successor node skeleton is 


constructed and Storage allocated to accommodate further information 


as it is obtained. 


Because of the strict requirement for mutually exclusive 


decision alternatives and event outcomes, a confirmation must be 


received from the decision maker that the successors just listed are 


indeed mutually exclusive. For decision nodes, the following questions 


confirm this fact: 


ARE THE ALTERNATIVES MUTUALLY EXCLUSIVE? 

DOES THE CHOICE OF ONE ALTERNATIVE EXCLUDE THE OTHERS? 

IS ONLY ONE CHOICE POSSIBLE? 

IS IT TRUE THAT CHOOSING ONE OF THESE ALTERNATIVES EXCLUDES 
THE OTHERS FROM BEING CHOSEN? 


AM I CORRECT IN ASSUMING THAT ONLY ONE CHOICE IS POSSIBLE? 
For event nodes, the following queries are typical: 


ARE THESE EVENTS MUTUALLY EXCLUSIVE? 

DOES THE OCCURRENCE OF ONE EVENT EXCLUDE THE OTHERS 
FROM HAPPENING? 

CAN JUST ONE OUTCOME HAPPEN AT A TIME? 


AM I CORRECT IN.ASSUMING THAT ONLY ONE CAN OCCUR? 


At this stage in the elicitation process, the preselected 
node has been completely expanded in the sense that all relevant 
information has been obtained about it in particular. It is now 
necessary to acquire enough information about the node’ Ss successors 
so that adequate rollback and sensitivity analysis can ‘Ge performed 
in preparation for the selection of another node to expand. The 
successors, as listed by the decision maker, are individually con- 


sidered by introducing them one at a time with short alert messages 
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followed by requests for provisional value, probability (if necessary), 
and cost. By iterating on each successor in this way, the decision 
maker is not forced to jump from one successor to another while trying 


to estimate values and probabilities, etc. The short alert messages are: 


LET'S CONSIDER "X" FOR A MOMENT. 
NOW CONSIDER "X". 

WHAT ABOUT "X"? 

LET'S TALK ABOUT "X" FOR A WHILE. 


Once the decision maker has been informed about the area of consid- 
eration, the first quantity to be requested is the provisional node 
value, that is, the worth to him of the possible opportunities that 
this particular situation may open. This value may be given in 
absolute terms, such as money, or in a relative utility scale 

that indiGates personal preference. The only criteria is consistency. 


Typical requests for provisional node value are: 


TRY TO PLACE A NUMERIC VALUE ON THIS SITUATION. 
WHAT VALUE WOULD YOU GIVE TO THIS OPPORTUNITY? 
HOW WOULD YOU EVALUATE THIS SITUATION? 

ESTIMATE THE VALUE IF YOU WERE IN THIS POSITION. 
WHAT IS THE POSSIBILITY WORTH TO YOU? 


If, by previous questioning, the selected node was determined 


to be an event, probabilities must be assigned to each of the successor 
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nodes. It is assumed that the decision maker is capable of providing — 
rough probability estimates without a complicated elicitation process. 


‘Typical queries for probability are as follows: | 


WHAT IS THE PROBABILITY OF THIS OUTCOME? 

WHAT ARE THE CHANCES THAT THIS EVENT WILL OCCUR? 
HOW MUCH OF A-CHANCE DOES IT HAVE? 
WHAT PROBABILITY WOULD YOU PLACE ON IT? 


Of course, no probability questioning occurs on the successors of 
decision nodes. Further, since the sum of the probabilities on all 
Successors must equal 1, no questioning is necessary on the last 
Successor. 7 | 

The final item of informationehat must be obtained for each 
of the successors is the cost. If the decision maker has indicated, 
_ through srevious questioning, that he pacha. stoiee a utility scale 
of his own choosing in order to express provisional values, it is 
assumed that any incurred costs will be absorbed in fie utility 
eres However, if he uses an absolute scale (aise money [13]) 
for representing provisional values, the sostof each decision 
alternative or event outcome is required. This cost is always assumed 
to be local and independent of the provisional yam. That is, the cost 
will remain constant even though the provisional value may change due 


to further refinement. When calculating rollback, the cost is incorporated 
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into the standard formulas as follows: 


where N is a decision node, N. are the successors of N, and C is the 


cost function. For event nodes, using the above notation: 
V(N) = £ (P(N, )CV(Ns)-C(Ny)3) 
i 
The following queries are typical of those used to elicit cost: 


WHAT IMMEDIATE COST IS EXPECTED? 

WHAT WOULD BE THE IMMEDIATE COST, ASSUMING THIS SITUATION? 
STATE THE COST IF THERE IS ONE. 

IF THERE IS AN ASSOCIATED COST, WHAT IS IT? 

WHAT IMMEDIATE COST IS ANTICIPATED? 


With the provisional values, probabilities, and costs obtained 
for each successor, the elicitation procedure is completed except for 
a possible option to perform an experiment. As described in detail in 
section III, the purpose of an experiment is to aid in estimating the 
probabilities of an event node. Of course, if the previous elicitation 
cycle was engaged in the analysis of a decision node, no request for an 
experiment is made. The following are typical questions that request 


for an experiment option: 
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‘CAN YOU IMPROVE THE PROBABILITY ASSIGNMENTS BY PERFORMING 
AN "EXPERIMENT"? | | 
ARE YOU UNHAPPY WITH THE ACCURACY OF THESE PROBABILITY 
ESTIMATES? es 
IS IT POSSIBLE TO DO ANYTHING IN ORDER TO IMPROVE THE 
ABOVE PROBABILITY VALUES? - 


If the option is declined, a new elicitation cycle begins by selecting 
a new node for expansion. If it is accepted, the experiment elicitation — 
process is initiated that begins with a request for the experiment | 


name and information concerning the time at which it is to be performed. 


WHAT TYPE OF EXPERIMENT COULD BE .PERFORMED? 
WHEN WOULD IT BE CONDUCTED? 


The decision maker is expected to respond ith che name of a previously - 
expanded internal node. The program searende the path from the root to 
the current tip under analysis and selects a node that is the closest 
match to the received response. Thus, it is not necessary for the 
decision maker to remember the vate name given. to each node. The 

tree is broken at this point with storage allocated to accomodate the 


new node. The next step is to ascertain the possible experiment outcomes: 


PLEASE LIST THE POSSIBLE OBSERVATIONAL OUTCOMES 
OF THE EXPERIMENT. 
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The final step in the experiment elicitation cycle is to ob- 
tain the event-conditional probabilities at the affected event node 
just expanded. With m event suteonas and n experiment observations, 
there are mn conditional probabilities. By using the fact that n sets 
of these probabilities must add to 1, the total is reduced to n(m-1) 
elicited probabilities. Each event outcome is taken in turn with each 


of the experiment outcomes as follows: 


SUPPOSE THAT C5" IS ABOUT TO HAPPEN... 
WHAT IS THE PROBABILITY OF "E," HAPPENING? 
WHAT IS THE PROBABILITY OF "E." HAPPENING? 


..etc. 


where C; is the jth event outcome and Ey; Eos etc. are the experiment 
observations. 

The request for experimentation completes the elicitation 
procedure for the expansion of a single node. The cycle of rollback, 
sensitivity analysis, node selection, and node expansion 1S repeated 
again and again until either all tip nodes have been removed from a 
status of “available for expansion" or the decision maker has requested 
ito prematurelyend the interview. At this time, the tree is closed and, 
after a final rollback calculation, he is made aware of the most 
promising scenario discovered. 

The summary information concerning the results of the interview 


is more than a simple indication of the best recommendation for the 


initial choice of alternatives. The decision maker is given a complete 


“plan of action" that will tel] him sat Geesicnt make at every 
decision node in the optimal solution subtree. This is that subtree 
which, starting from the root, takes the maximum-valued branch at each 
decision node and alZ branches at each event node. In addition, 
experiment nodes are given special consideration. Each decision that 
comes under the influence of the experiment is mentioned separately 
with recommended actions for each observational outcome so that the 
decision maker may obtain the most information from the experiment. 
The following flowchart (Figure 4.2) diagrams the complete elicitation 


procedure in a compact form and describes its overall logical structure. 
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Figure 4.2. Elicitation Procedure Flowchart 
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‘Figure 4.2. Elicitation Procedure Flowchart (Continued) 
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Figure 4.2. Elicitation Procedure Flowchart (Continued) 
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V. SAMPLE INTERVIEW SESSION 


The following hypothetical situation was chosen as an example 

for demonstrating the program operation. We imagine a scientist who 
1s facing the dilemma of sending his brilliant proposal either to. 
governmental agency X or to agency Y. (The example is not altogether 
unrealistic from the environment surrounding the funding of thts 
research.) It is assumed, because of inter-agency code of conduct, that 
he cannot send the proposal to both agencies simultaneously. Agency Y 
has already indicated interest in his work and a willingness to support 
it at a level of $50,000. Agency X, on the other hand, has not yet had 
an opportunity to appraise it and further, is not committing funds 
until the end of the year (i.e. two months hence). The potentially 
available funds from agency X are much higher: full funding at a 
level of $100,000 or partial funding at $70,000 (see Figure 5.1). 

The basic decision that the scientist must make at the 
moment is whether to send the proposal to agency Y immediately or to 
hold it for two months for the purpose of improving it to fit some of 
the specific needs of agency X. After the two months has elapsed, the 
opportunity to send it to agency Y is still open. However, some 
erosion in certainty would result due to the delay in submission. 

During the elicitation of the probability estimates 

concerning the funding from agency X, the scientist feels uncertain 
about the values he reports. He realizes that the delay in submission 


will also offer him the opportunity to gain information by soliciting 
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the opinion of an official from agency X as to the technical 
compatibility of the proposal with respect to the overall agency 
objectives. The official cannot be expected to give more than a 
"favorable" or "adverse" response. Through past experience, the 
scientist is aware of the official's tendency to respond over- 
optimistically even regarding proposals which are not about to be 
funded. This tendency can be formulated in terms of the event-conditional 
probabilities of the experiment associated with evaluating the proposal. 
The following interview session demonstration is reproduced 
directly from the computer output. An asterisk is the signal from the 
computer that it 1s ready to receive input from the user. Thus, text 
immediately following an asterisk is considered to have been typed 
by the scientist. Following the interview are two figures showing the 
resulting decision Structure. Figure 5.2 shows all of the duplicated 
subtrees due to the experiment and Figure 5.3 shows the condénsed 


structure using experiment nodes: 


ae 


WHAT’S YOUR PROBLEM? *SENDING THE PROPOSAL TO AGENCY Y OR WAITING 
"PLEASE LIST THE ALTERNATIVES THAT YOU HAVE, ONE AT A TIME. 
_1.*SUBMIT TO AGENCY Y IMMEDIATELY 

_ 2.*WAIT FOR TWO MONTHS 

3.*NO MORE 

ARE THE ALTERNATIVES MUTUALLY EXCLUSIVE? 

*YES | 

NOW CONSIDER “SUBMIT TO AGENCY Y IMMEDIATELY’, | 
ASSUMING THAT THE CURRENT SITUATION WERE TRUE, TRY TO ESTIMATE 
HOW GOOD IT IS INDEPENDENTLY OF OTHER POSSIBILITIES. YOU MAY USE 
MONEY AS A SCALE OR ONE OF YOUR OWN CHOOSING, HOWEVER, YOU MUST 
BE CONSISTENT AND USE THE SAME SCALE THROUGHOUT THE INTERVIEW. 
DO YOU WISH TO USE MONEY AS A SCALE? 

*YES | 

ENTER YOUR ESTIMATE, 

50000 

ARE THERE ANY HIDDEN COSTS NOT TAKEN INTO ‘ACCOUNT IN THE 
PREVIOUS VALUE ESTIMATION? 

*NO 

LET'S LOOK AT “WAIT FOR TWO MONTHS” 

WHAT VALUE WOULD YOU GIVE TO THIS SITUATION? 

~*70000° 


WHAT IMMEDIATE COST IS EXPECTED? 
*O | 


a & 


SUPPOSE THAT YOU HAD CHOSEN TO WAIT FOR TWO MONTHS 

CAN YOU THINK OF THINGS THAT MAY HAPPEN AS A RESULT? 

*NO 

IS THERE A DECISION TO BE MADE AT THIS POINT? 

*YES 

STATE THE CHOICES THAT YOU HAVE, 

1.*SEND TO X 

2.*SEND TO Y 

3. *DONE 

DOES THE CHOICE OF ONE ALTERNATIVE EXCLUDE THE OTHERS? 
*YES 

NOW CONSIDER “SEND TO xX” 

TRY TO PLACE A NUMERIC VALUE ON THIS SITUATION. 

*70000 | 

WHAT WOULD BE THE IMMEDIATE COST, ASSUMING THIS SITUATION? 
6) | | 

LET'S LOOK AT “SEND TO Y” 

HOW WOULD YOU EVALUATE THIS SITUATION? 

*50000 

HOW MUCH EXPENSE IS ANTICIPATED? 

*0 

ASSUMING THAT “SUBMIT TO AGENCY Y IMMEDIATELY” WAS PICKED, 
ARE EVENTS ABOUT TO HAPPEN OVER WHICH YOU HAVE NO CONTROL? 
*NO 

DO YOU HAVE A CHOICE OF ALTERNATIVES? 

*NO 
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DO YOU WISH TO STOP EXPLORING FURTHER IN THIS DIRECTION? 
ie | | 
SUPPOSE THAT “SEND TO Y" IS YOUR CHOICE, 

ARE THERE SOME EVENTS THAT MAY HAPPEN? 

*YES 

PLEASE LIST THE OUTCOMES, 

L.* ACQUIRE FUNDING FROM Y 

2.*Y REJECTS PROPOSAL 

3. *DONE 

ARE THESE EVENTS MUTUALLY EXCLUSIVE? 

*YES 

NOW CONSIDER “ACQUIRE FUNDING FROM Y” 

ESTIMATE THE VALUE IF YOU WERE IN THIS POSITION. 
*50000 ; 


TRY TO ESTIMATE THE PROBABILITY THAT THIS EVENT WILL HAPPEN, 
*0.8 | 


IF THERE IS AN ASSOCIATED COST, WHAT IS IT? 
*0 | 
LET'S LOOK AY “Y REJECTS PROPOSAL” 

WHAT IS THE SITUATION WORTH TO YOU? — 


x0 


STATE THE IMMEDIATE COST IF THERE IS ONE. 

*Q an 

CAN YOU IMPROVE THESE PROBABILITY ASSIGNVENTS BY 
PERFORMING AN EXPERIMENT? 

*NO 
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SUPPOSE THAT "ACQUIRE FUNDING FROM Y” HAPPENED, 

ARE THERE SOME OPTIONS AVAILABLE TO YOU? 

*NO 

COULD UNCONTROLLABLE OUTCOMES OCCUR? 

*NO 

HAS THERE BEEN ENOUGH DETAIL EXPRESSED SO FAR? 

*YES 

WHAT IF "“Y REJECTS PROPOSAL” OCCURS? 

WOULD THERE BE OPPORTUNITIES OPEN TO YOU NOW? 

*NO 

CAN YOU THINK OF THINGS THAT MAY HAPPEN AS A RESULT? 
*NO | 

SHALL WE TERMINATE THE INTERVIEW? 

*NO | 

WHAT IF YOU CHOOSE TO SEND TO xX? 

ARE EVENTS ABOUT TO HAPPEN OVER WHICH YOU HAVE NO CONTROL? 
*YES 

EXACTLY WHAT EVENTS COULD OCCUR? 

1.*FULL FUNDING FROM X 

2.*PARTIAL FUNDING FROM X 

3.*X REJECTS PROPOSAL 

4.*NO MORE 

DOES THE OCCURRENCE OF ONE EVENT EXCLUDE THE OTHERS FROM HAPPENING? 
*YES 
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NOW CONSIDER “FULL FUNDING FROM X” 

WHAT VALUE WOULD YOU GIVE TO THIS SITUATION? — 
#100000 < 
WHAT ARE THE CHANCES OF THIS OUTCOME HAPPENING? 
*0.4 ~ | 
WHAT IMMEDIATE COST IS EXPECTED? 

*0 

LET'S LOOK AT “PARTIAL FUNDING FROM X” | 

TRY TO PLACE. A NUMERIC VALUE ON THIS SITUATION, 
*70000 

WHAT ARE THE CHANCES THAT THIS EVENT WILL OCCUR? 
*0.4 — 

WHAT WOULD BE THE IMMEDIATE COST, ASSUMING THIS SITUATION? 
*0 | a 

WHAT ABOUT “X REJECTS PROPOSAL’ ? 

HOW WOULD YOU EVALUATE THIS SITUATION? 

*0 | | 

HOW MUCH EXPENSE IS ANTICIPATED? 

*O 

ARE YOU UNHAPPY WITH THE ACCURACY OF THESE PROBABILITY ESTIMATES? 
*YES | 
WHAT EXPERIMENT COULD BE PERFORMED? — 

*EVALUATE THE PROPOSAL | 

WHEN WOULD IT BE PERFORMED? 

*AFTER WAITING TWO MONTHS _ 


ey 


PLEASE LIST THE POSSIBLE OBSERVATIONS FROM THE EXPERIMENT. 
1.*FAVORABLE RESPONSE 

2.*ADVERSE RESPONSE 

3. * DONE 

CAN JUST ONE OUTCOME HAPPEN AT A TIME? 

*YES 

SUPPOSE THAT "X REJECTS PROPOSAL” IS ABOUT TO HAPPEN... 
WHAT IS THE PROBABILITY OF "FAVORABLE RESPONSE” HAPPENING? 
*0.3 

SUPPOSE THAT “PARTIAL FUNDING FROM X” IS ABOUT TO HAPPEN... 
WHAT IS THE PROBABILITY OF “FAVORABLE RESPONSE” HAPPENING? 
*0.9 

SUPPOSE THAT “FULL FUNDING FROM X° IS ABOUT TO HAPPEN... 
WHAT IS THE PROBABILITY OF “FAVORABLE RESPONSE” HAPPENING? 
*1 

ASSUME THAT “FULL FUNDING FROM X” HAS HAPPENED. 

IS THERE A DECISION TO BE MADE AT THIS POINT? 

*NO | 

ARE THERE SOME EVENTS THAT MAY HAPPEN? 

*NO oe 

PERHAPS WE SHOULD TALK ABOUT SOMETHING ELSE? 

*VES | 
LET US SAY IT WAS “PARTIAL FUNDING FROM X° THAT ACTUALLY TOOK PLACE, 
DO YOU HAVE A CHOICE OF ALTERNATIVES? 

as | 
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COULD UNCONTROLLABLE OUTCOMES OCCUR? 
*NO | 
SHALL WE TERMINATE THE INTERVIEW? 
*YES 7 
CONCERNING SENDING THE PROPOSAL TO AGENCY Y OR WAITING — 
YOUR BEST DECISION IS TO WAIT FOR TWO MONTHS 
EVALUATE THE PROPOSAL AND, .+ | 
IF "FAVORABLE RESPONSE” IS OBSERVED, SEND TO X 
IF “ADVERSE RESPONSE’ IS OBSERVED, SEND TO Y 
PROGRAM FINISHED 
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Figure 5.2. The Full Decision Tree 


LS 


Ny 
$68,000 | 
$79 , 800 &, 
$40 ,000 + 


$50,000 


Figure 5.3. The Condensed Decision Structure 
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VI. PROGRAM DESCRIPTION 


An overall diagram of the flow of information in the 
program is shown in Figure 6.1. The decision nakey communicates with 
the system through a remote terminal display. Information is viewed 
On the screen and responses are typed at the console keyboard. Ques- 
tions and answers are monitored by the "Query/Response Control" 
subsystem. It is responsible for activities such as message printing, 
information display, keyword analysis of input, etc., and the 
interface control between the decision maker and the main processing 
system. The "Tree Expansion Control" mechanism activates and maintains 
the primary tree expansion cycle of (1) rollback, (2) sensitivity 
analysis, (3) node selection, and (4) node expansion. Of these four 
major steps, the decision maker is only aware of the fourth, "node 
expansion’, which guides him through the elicitation procedure. 

Each of the four major step-subsystems has access to the decision 
tree data but only the node expansion subsystem can alter it. 

The program is implemented in the LISP programming lan- 
guage [14,15] on a Digital Equipment Corporation PDP-10 Computer [16] 
located in the Center for Computer-Based penavioral Studies (CCBS) 
in Franz Hall on the UCLA Campus. The particular version of LISP that 
was used is called "Irvine LISP" C173 from the University of Calif- 
ornia at Irvine. It is a modified and enriched version of "Stanford 
LISP" [18]. A concerted effort was spent in an attempt to use as few 


sophisticated and version-dependent functions and special features 
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~ Figure 6.1. Program Organization 


as possible. Thus, most of the program contains elements found in 
the intersection of all major LISP dialects. This allows the program 
to be peanereuned to another computer with a minimum of translation 
effort. The program itself contains a little over 100 defined 
functions and occupies almost 600 lines of printed LISP code. It 
does, however, execute rapidly as there is no detectable delay in 
response time. A complete program code listing can be found in the 
Appendix. 

The basic data elements provided by the LISP programming 
language eoneiet of atomic items such as numbers and words plus a 
Single structural item called a binary cell. Each data element can 
be referenced by any number of "pointers"; words can, themselves, 
reference one other item; and binary cells can reference two items 


as shown below. 


ie! Naf 


WORD 


The data elements of the decision trees have been built from the 
above LISP primitive elements. The nodes and arcs of the tree are 


considered to be individual entities that are connected by pointers. 
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| For example, two nodes connected by an arc areusually drawn as follows: 


The above structure is, however, represented in data storage as 


follows: 
5 


6) 


Each arc is doubly connected to both its single predecessor and 
Sui successor node. Similarly, each node points to its 
predecessor arc and to all of its successor arcs as well as being 
pointed to by them. | 
- The internal structure of ach ite permits the storage 
of necessary information for computational processing (see Figure 6.2). 
Each node has 9 data items associated with it. The NAME of the node 
is a list of words received from the decision maker. One binary cell 
is needed for each word in the node name. The BEST PATH FLAG is set 


to "true" when the node is found to be in the optimal solution plan 
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Figure 6.2. Internal Node Structure 


after rollback calculations are completed. The PREDECESSOR ARC 
pointer connects directly to the arc structure that leads from the 
predecessor node. Similar provision is made for connection to. 
multiple SUCCESSOR ARCS. The node TYPE is stored as either an integer 
from 0 to 3 or NIL indicating an unknown type. The following table 


lists the possible node types and their corresponding codes: 


CODE NODE TYPE 
NIL unknown 

0 termina! 

] decision 

2 event 

3 experiment | 


A newly created node has type NIL until further information is 
acquired. The VALUE item is determined by the rollback procedure 
and may be a list of numbers if it is a collapsed node due to an 
experiment. Likewise, the SENSITIVITY is calculated for each 
value. The EXPANDED FLAG is set to "true" only when complete 
information about a node has been obtained. The PROCESS FLAG is an 
internal status indicator that is used by the rollback functions. 
The internal structure of an arc parallels that of a node. 
Figure.6.3 shows this structure in detail. The data items are similar 
except for the SUCCESSOR NODE which connects to one node only. The 
PROBABILITY (which may be a vector) and the COST are obtained from 


the decision maker. 
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Figure 6.3. Internal Arc Structure 


. 

‘The node and arc structures are very flexible. All important 
cctémation can be stored either at the node or on the arc (branch) , 
wherever it is most natural. The tree can be searched either forward 
or backward as required. In addition, a "“super-structure’ of binary 
cells connects all nodes together as they are being created (see 
Figure 6.4). A similar one connects all arcs. These super-structures 
are used when node or arc processing is necdeu that is independent 
of tree structure. 

The defined program functions are divided into four 
general categories: (1) data functions, (2) manipulation functions, 
(3) input/output functions, and (4) processing functions. The data 
functions are the lowest level functions and allow storage and 
retrieval from the data structures for node and arc information. 
The LISP language has perfectly good data functions of its own for 
this very same purpose. However, by defining new ones, two | 
Important advantages are gained: data independence and program 
readability. Data independence is gained when it is possible to 
change the data structure without changing the applications programs. 
‘When a second level of data functions is defined in terms of the 
primitive LISP functions, only their definitions need to be altered 
when a modification of data structure is required. Secondly, English 
words that reflect the meaning of the function can be used as names, 
thus making the program more readable. The following is a Synopsis of 
the data functions and a short phrase describing the data item(s) 


that each retrieves. 
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Figure 6.4. Node and Arc Super- Structures | 


NAME(X) 
NAMES(L) 
PUTNAME(X.V) 


FINDANAME(N) 


FINDNAMES (L) 
NAME; (N) 


BP(X) 
PUTBP(X,V) 
PRED(X) 
PUTPRED(X,V) 


PREDNODE (N) 


SUCC (X) 


PUTSUCC(X,V) 


SUCCNODES (N) 
ADDSUCC(N,V) 


TYPE(N) 
PUTTYPE(N,V) 
TYPEO(N) 
TYPE1(N) 
TYPE2(N) 
TYPE3(N) 


The words comprising the name of node X or arc X. 
NAME applied to each node or arc in list L. 
Stores a new name V at node X or are X. 

Searches node N or its predecessors for a name. 
FINDANAME applied to all nodes in list L. 


Attaches a semi-colon to the end of the name of node N. 


Contents of the BEST PATH FLAG cell. 
Places value V into BEST PATH FLAG cell of node or arc X. 


Predecessor arc of node X or predecessor node of arc X. 
Stores V as a predecessor to node X or arc X. 


PRED( PRED(N) ) 


Successor arcs to node X or successor node to arc X. 
Stores V as a new successor to node or arc X. 
SUCC(SUCC(N)) 


Adds a successor arc V to node N. 


Numeric type of node N. 

Stores V as type of node N. 

"true" if type of node N is O; NIL otherwise. 
"true" if type of node N is 1; NIL otherwise. 
"true" if type of node N is 2; NIL otherwise. 


"true" if type of node N is 3; NIL otherwise. 


a 


VALUE(N) 

PUTVALUE(N,V) 
VALUEO(N) 
VALUES(L) 
ADDVALUE(N,V) 


SEN(N) 
PUTSEN(N,V) 
ADDSEN(N,V) 


EXP(N) 
EXPS(L) 
PUTEXP(N,V) 
FLAG(X) 
PUTFLAG(X,V) 
FLAGS(L) 


PROB (A) 
PUTPROB(A,V) 
ADDPROB(A,V) 
PROB1 (A) 
PROBS (L) 


PROBIS(L) 


Provisional value of node N. 

Stores value V at node N. 

Applies VALUE to node N; if none stored, returns 0. 
VALUE applied to all nodes in list L. 


Adds value V to those already at node N.- 


Sensitivity of node N. 
Places sensitivity value V at node N. 


Adds sensitivity value V to those already at node N. 


EXPANDED FLAG cell contents. 


EXP applied to all nodes in list L. 


| Stores value V in EXPANDED FLAG cell. 


Contents of PROCESS FLAG cell. 
Stores V into PROCESS FLAG cell. 


FLAG applied to all nodes or arcs in list L. 


Probability of arc A. 
Stores probability V at arc A. 


Adds probability V to those already at arc A. 


Applies PROB to arc A; if none stored, returns 1. 


PROB applied to all arcs in list L. 
PROB] applied to all arcs in list L. 
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—COST(A) Cost of arc A. . 
PUTCOST(A,V) «Stores cost Vat arc A. 


COSTO(A) | Applies COST to arc A; if none stored, returns 0. 


COSTS(L) COST applied to all arcs in list L. 


The manipulation functions ane defined to transform list 
structures without any particular application dependency. The LISP 
language has many available manipulation functions and relatively 
few new ones needed to be defined. Some of the functions operate 
on arbitrary lists and shee are tailored for the specific 


decision tree structure. The definitions can be found in the Appendix. 


FLAT(L This function removes any hierarchy that may exist in the 
argument list L and returns a one-level list containing all] of the 
atomic data items of the original list in their proper order. The 


scan algorithm searches depth-first from left-to-right. 


SINGLE(L) This function removes all duplications of atomic items | 


from a list at the top level only. 


MEMBLISTV(L1,L2) This function returns "true" if any member of the 


list Ll is also a member of list L2. 
MEMBLIST&(L1,L2) Similarly, this function returns "true" only if aZZ 
members of list LI are also members of list L2. If any elements of 


list LI are themselves lists, MEMBLISTV is called on. those and vice 


versa for MEMBLISTZ. Thus, the two functions are mutually recursive. 


BLEND(F V PC) For any function F, BLEND returns a list that is 

the computation F( Ps(V.-C, ) ) where P. are probabilities in list P, 
V; are values in list V, and C; are costs in list C. This function 

is a useful way of computing rollback for different types of nodes 


uSing the same code. 


ALLATOMS(L) This functions returns "true" only if all of the 


members of list L are atomic data items. 


CONSTANT(S) A self-referent node pointing to S is formed when the 
function CONSTANT is used. 


\ 


The reason that such a function is necessary stems from the pre- 
defined nature of the LISP function MAPCAR [17]. The function MAPCAR 
applies any specified function of one argument to the members of a 
given list (one at a time) and returns a list of results. If MAPCAR 


is given more than one list upon which to iterate, functions of more 
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— than one argument may be supplied. In these cases, the elements of the 
multiple lists are taken in parallel until one of them runs out ie 
which time MAPCAR halts. In many instances, it is desirable to give 

a constant argument to the specified function. Rather than creating 

a list of duplicates, a self-referent "list" fools MAPCAR into 


believing that an infinitely long list exists. 


BAYES(A,B) Bayes’ rule is used in this function to calculate 


observation-conditional probabilities. 


FIRSTNUM(L) A simple function that returns the first number found 


in the argument list L. 


STAT() The function STAT is a debugging tool that gives a status 


report on the values stored in the entire decision tree. 


DUMP() The final summary report to the decision maker is initiated 


with this function. 


MAXNODE(L) The node with the highest current provisional value 


in list L is returned as the value of MAXNODE. 


TIPS(N) As might be expected, the function TIPS returns a list 


of tip nodes using node N as the root of a subtree. 
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The input/output functions control all of the interaction 
between the program and the decision maker. They consist of 
pre-defined messages (labelled M1 through M17), question functions 
(labelled with a prefix of "Q"), and various input/output processing 
functions. The message functions simply cycle through a list of 


equivalent messages printing a new one out each time it is needed. 


M1 Alert (decision) message. 

M2 Alert (event) message. 

M3 Determine node type (decision). 

M4 Determine node type (event). 

M5 Determine node type (terminal). 

M6 Elicit decision alternatives. 

M7 Elicit event outcomes. 

M8 Decision alternatives mutually exclusive? 
M9 Event outcomes mutually sxciusiver | 

M10 Successor iteration alert message. 

M11 Elicit provisional value. | | 

M12 Elicit probability. 

M13 Elicit cost. 

M14 Request for experiment. 

M15 Request for experiment name. 

M16 Request for experiment time. 

M17 Elicit experiment conditional probabilities. 
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The question functions use the message functions and are 
labelled to refer to them. For example, question Q345 references 


message functions M3, M4, and M5. 


QYN Requests a "yes" or a "no" response. 


Q345 Elicits node type. 
Q67 Elicits decision alternatives or event outcomes. 
—Q89 Requests confirmation for mutually exclusive alternatives 


or outcomes. 


QT] Controls elicitation of provisional value. 
QI2 Controls elicitation of probabilities. 
Q13 Controls elicitation of cost. 


The following input/output functions perform printing and reading 


chores, etc. 


PR(LF L) ~~ The function PR prints atoms in a list with a trailing 
line-feed character if the variable LF is "true". Thus, it js 
possible to print messages from more than one place in the program 


on the same line. 


OUT(V CL) The function QUT controls the cycle of messages and 


keeps track which message. of a set is next to print. 


GETINPUT() This is the only function that actually requests 


an input. from the decision maker. It prints an asterisk as a signal. 


OT. 


The processing functions are the main control over tree 
construction and the elicitation procedure. They use all of the 


functions described thus far. 


START() The top level function that is called to begin the program 
is START. It initializes all of the system variables and expands 
the root node. It then calls START* which begins the elicitation 
cycle. When the interview is completed, START evokes the summary 


function DUMP which gives the final results to the decision maker. 


START*(N) The function START* controls the continuous cycle of 
rollback, sensitivity analysts, node selection, and node expansion. 
START* completes it job when the decision maker wishes to end the | 
interview or when the node selection function indicates that all 


tree nodes have become terminal. 


ROLLBACK() This function controls the rollback procedure and 


calls ROLLCALC for computation on individual nodes. 


ROLLCALC(N) The rollback value of node N is calculated and stored 


in the internal structure representing the node itself. 
SENSITIVITY() | This function controls the sensitivity analysis 


for the entire tree. It calls SENCALC for the computation on a 


single node. 
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SENCALC(N) This function calculates the sensitivity value for the 
specific node argument N and stores it in the appropriate location 


for use by the node selection function. 


NODE-SELECT(R) The node selection function chooses a new node for 
expansion out of the subtree with root node R by comparing the 


sensitivity values. 


EXPAND(N) The function EXPAND is the heart of the program because 

it directly controls the logic for the elicitation procedure. It 

evokes other functions that print alert messages; elicit provisional 
values, probabilities, costs, etc.; request responses from the decision 


maker; and allocate storage for new nodes and arcs. 


EXPERIMENT(N) This function is called by EXPAND when an experiment 
is desired by the decision maker. It controls the elicitation process 
for experiment nodes and inserts the node structure into the existing 


tree at the proper place. 


FINDEXPNODE(F N) This function aids EXPERIMENT by locating the 
place of insertion of an experiment node. The method used is keyword 
matching of the words comprising the node names given by the decision 


maker with the name of the experiment. 
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VII. CONCLUSIONS 


Although the program has not been in operation for a 
sufficient length of time to permit exhaustive tests, it was found 
that the elicitation system could provide an adequate and useful 
tool for decision aiding in those applications that naturally lent 
themselves to decision tree representations. The most desirable 
features were (1) the ease of operation that resulted once familiarity 
was gained with the mechanics of interaction; (2) the inherent 
"direction" of elicitation that leads to the exploration of the most 
important and appropriate areas for quickly resolving the major 
decision problem with few questions; and (3) the determination to 
maintain, as closely as possible, a one-to-one correspondence 
between the internal storage representation of events and the 
decision maker's perceptual representation. 

This latter property is considered to be the most important 
and points out two observed deficiencies that will now be discussed 
in detail. The decision tree jensnde mutuariy exclusive decision 
alternatives and event outcomes. Thus, it is always necessary to 
coneien this fact by continually asking the decision maker to check 
his responses. This confirmation is necessary because of the ease with 
which the non-technical user may misinterpret the system's queries. 
It is very tempting to report aspects of the previously expanded node 
rather than new future situations. For example, suppose that the 
decision maker has just given the following alternatives to a 


decision node: 
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Assume that, through sensitivity analysis, it has been determined 
that tip node t is the most promising to expand. Further, the decision 
maker has indicated that t is an event node. Upon the request TOY 


the event outcomes of node t as follows: 
EXACTLY WHAT EVENTS COULD HAPPEN? 
the decision maker responds with this set of items: 
1.* BUY A HOUSE 
2.* GET AN APARTMENT 
3.* RENT A ROOM 
These responses constitute mutually exclusive event outcomes as 
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required by the decision structure and may be followed by the 


elicitation of probabilities. However, consider the following set: 


we | 


.* BUY A CAR 
2.* GET AN APARTMENT 


ww 


.* FIND A JOB 


These responses seem to follow just as naturally from the elicitation 
query and yet they are not mutually exclusive. Indeed, they are aspects 
or attributes of the previous decision alternative. 

It would be desirable to incorporate these responses into 
the overall decision tree as a new type of node. There are, however, 
two distinct interpretations of the meaning of the outcomes and a 
separate query would be needed to distinguish them. The first inter- 
pretation is that they are a description of the event ands therefore, 
will occur with absolute certainty and serve primarily to identify 
the event. The reason that the decision maker might be tempted to 
provide a list of attributes in response to event queries is that he 
focuses on the node aspects as auxilliary tools for mental estimation 
of value (also called value dimensions). The second interpretation is 
that all possible combinations of the mentioned outcomes and their 
complements could be realized. In this case, the event node can be 
represented as a series of sequentially ordered events as shown in 
Figure 7.1, where Pa is the probability of event 7 occurring and 


P. = noes stands for the probability of event 7 not occurring. 
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One immediate problem with this formulation is that the 

structure forces a duplication of event nodes as well as imposing 
a time ordering that was not implied by the responses. An alternate 
structure iS shown in Figure 7.2 which requires only one node with 
more branches representing a set of mutually exclusive event com-. 
binations. A separate branch is needed for each path in the original 
structure. The probability assigned to each branch is the product 
of the probabilities along the path leading to the corresponding 
combination of events. This representation, however, introduces its 
own problems. Each tip node now stands for a combination of events 
and the property of one-to-one correspondence between events and 
nodes is lost. This property is one of the primary motivations for 
collapsing duplicate subtrees by using a single experiment node. In 
manual elicitation, the analyst can usually utilize his real-world 
knowledge to resolve these ambiguous situations. 

A similar situation arises when non-mutually exclusive 
decision alternatives are encountered. Suppose, for example, that 


the decision maker responds with the following set of alternatives: 
1.* INVEST IN STOCK A 
2.* INVEST IN STOCK B 


3.* INVEST IN STOCK C 


It seems clear that unless there is a minimum purchase requirement, 


any combination of the stocks could be chosen for investment purposes 
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Figure 7.2. Order-independent, Non-duplicative Event Node 
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making the reported alternatives noncmitualty exclusive. In fact, 
when considering the amount of capital invested, they form the 
boundary of a continuous action set. 

The second deficiency iS a subtle problem for any automated 
elicitation system and surfaces as a result of representing knowledge 
in tree form. In many real-world applications, the decision maker may 
not perceive a problem in the form of a time sequence of decision 
alternatives and event outcomes, but rather as a static network of 
influences surrounding ztssues.* Consider, for example, our perception 
of the environmental pollution problem. The issues of capital 
investment, energy needs, energy supply, unemployment, public health, 
etc. all seem to be tightly interwoven in a network of cause-and-effect 
relationships. The main difficulty in attacking such a problem seems | 
to be that of unraveling the underlying causal network (on the basis 
of insufficient data) rather than that of planning with action/event 
scenarios. The major difference in the formal representation required 
for such problems and the one handled by decision trees is that the 
atomic entities admitted by the latter representation are restricted 
to be descriptions of "world states" or decision situations. The 
decision maker can express relations among these situation units but 
is unable to express relations between the constituents of these units. 
For example, he may assess the value of a certain situation consisting 
of a combination of attributes but he is unable (with the present 


system) to express the relative value of each individual attribute 


—*Ward Edwards elucidated this point in personal discussions with him. 
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or cross impacts among them. Likewise, when the decision maker is asked 
to assess the iikersneod of a certain event taking place, he ought to 
consider the entire state of affairs prior to that occurrence. He 

cannot, for example, express the belief that pollution is a positive | 
contributor to ence independently of other situational factors 
such as unemployment or some energy embargo. | 

To facilitate an “issue-oriented" problem elicitation 
_ program, the internal machine representation of problem eau eione 
should follow a different structure. In the Artificial faceieceaesr 
literature, such structures are referred to as “problem reduction" 
representations [8]. Each node in a tree represents a sub-problem 
or a sub-goal rather than a state description. The resolution of each 
"issue", which the decision maker perceives as part of his problem, 
constitutes a reduction of the global problem into several components. 
These can be further reduced to their constituencies, and so on. The 
tree expansion procedure guides the decision maker in selecting the 
‘issue to explore next rather than the time-sequenced scenario that 
he should follow next. _ 7 | 

It is believed that an amalgamation of the issue-oriented — 
and the scenario-oriented approaches into a unified problem repre- 
sentation would yield greater matching between human perception and | 
organization of information and would render computers a more - 


effective decision aid to man. 
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APPENDIX 


PROGRAM CODE LISTING 
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obit 


monn ; ; 
Ow CON mew h 


ome’ 
— 


(SETQ BASE 12)(SETQ IBASE 12) 
(DE NAME(X)(CAR X)) 
(DE NAMES(L)(MAPCAR( FUNCTION NAME)L)) 
(DE PUTNAME(X V)(CAR(RPLACA X V))). 
(DE FINDANAME(N) (COND 
((NAMB W)) | 
(C(NULL(PRED N)) (QUOTE RGOT)) 
((NAME(PRED N))) 
(TC QUOTE NONE)) )) 
(DE F INDNAMES(L) (MAPCAR( FUNCTION FINDANAME)L)) 
(DE NAME; (N)(LISTC(NAME N)5 )) 
(DE BP(N)(CADR N)) 
(DE PUTBP(N V)(CAR(RPLACA(CDR N)V))) 
(DE PRED(X)(CADDR X)) 
(DE PUTPRED(X V)(CAR(CRPLACA(CDDR X)V))) 
(DE PREDNODE(N)(PRED(PRED N))) 
(DE SUCC(X)(CADDDR X)) 
(DE PUTSUCC(X V) (CAR(RPLACA(CDDDR X)V))) 
(DE SUCCNODES(N) (MAPCAR(FUNCTION SUCC)(SUCC N))) | 
(DE. ADDSUCC(H V) (CAR(RPLACA(CDDDR N) (CONS vcsuce W))))) 
(DE TYPECN) (CAR(CDDDDR Ni) )) 


(DE PUTTYPE(N V) (CARCRPLACA (CDDDDR N)V))) 


(DE TYPEO(N)(EQ(TYPE N)O)) 


(DE TYPE1(N)(EQ(TYPE N)1)) 


(Di TYPE2(N)(EQ(TYPE N)2)) 

(DE TYPE3(N)(EQ(TYPE W)3)) 

(DE VALUE(N)(CADR(CDDDDR N))) 

(DE PUTVALUE(N PC CAn Ge bine Ce nCREDDR N))V))) 
(DE VALUEO(N)(COND( (VALUE N))(T 0))) | 

(DE VALUES(L)(MAPCAR(FUNCTION VALUEO)L)) 


Chin. 


(DE 


(DE 
(DE 
(DE 


(Di 
(DE 
(Di 
(DE 
(DE 
(DE 


(DE 


(DE 
(DE 
(DE 
(DE 


(DE 
(DE 
(DE 
(DE 
(DE 
(DE 
(DE 
(DE 


ADDVALUE(N V)(PROGN | 
(COND((ATOM(VALUE N))(PUTVALUE N(LIST(VALUE N 
~—(T(PUTVALUE NCREVERSEC(CONS VCREVERSE(SEN N))) 

SEN(N)(CADDR(CDDDDR N))) 

PUTSEN(N V) (CAR(RPLACA(CDDR(CDDDDR N))V))) 

ADDSEN(N V)(PROGN 
Hohl pcieh o toblgaie panes N)) (RUTSEH N(LIS TCSEN 4)¥) 
(TC PUTSEu WOR LoL, (CONS Vine 3H rOL ese _ i ) 

EXP (iV) (CADDDR(CDDDDE N))) — 

EXPS(L)(MAPCAR(FUNCTION EXP)L)) © 

PUTEXP(N V)(CAR(RPLACA(CDDDR(CDDDDR N))V))) 

FLAG(X)(CAR(LAST X))) 

PUTFLAG (X V) (CAR(RPLACA(LAST X)V))) 

FLAGS(L)(MAPCAR(FUNCTIOWN FLAG)L)) 

OBN(A)(CADR A)) 

PUTOBN(A V)(CARC(RPLACA(CDR A)V))) 

PROB(A)(CAR(CDDDDR A))) 

PUTPROB(A V) (CAR(RPLACA(CDDDDR A)V))) 

ADDPROB(A V)(PROGN 
(COND((ATOM(PROB A))(PUTPROB A(LIST(PROB A)V 
(T(PUTPROB ACREVERSE(CONS VCREVERSE( PROB A)) 

PROB1(A)(COND( (PROB A))(T 1))) 

PROBS(L) (MAPCAR( FUNCTION PROB)L)) 

PROB1S(L)(MAPCAR(FUNCTION PROB1)L)) 

COST(A)(CADR(CDDDDR A))) 

PUTCOST(A V)(CAR(RPLACA(CDR(CDDDDR A))V))) 

COSTO(A)(COND( (COST A))(T 0))) 

COSTS(L) (MAPCAR( FUNCTION COSTO)L)) 

FLAT(L)(PROG(R) 

(FLAT* L) 


Nene Oe” 


id), 
))) 


es << 


ELL 


(RETURN(REVERSE R)) )) 
(DE FLAT*(L) (COND 
( (NULL ghia 
((ATOM L)(SETQ R(CONS L R))) 
(TCPROGN 
 (FLAT*(CAR L)) 
(FLAT*(CDR L)) )))) 
(DE SINGLE(L) (COND 
((NULL L)NIL) 
((MEMBER( CAR L)(CDR L))(SINGLE(CDR L))) 
(T(CONS(CAR L)(SINGLE(CDR L)))))) 
(DE FIRSTNUM(L) (COND 
((FIRSTNUM*® L)) 
(T L) )) | 
(DE FIRSTNUM*(L) (COND 
((NULL L)WNIL) 
((NUMBERP(CAR L))(CAR L)) 
(T(FIRSTNUM* (CDR L))))) 


(DE SETOQS() (PROGN 


(SETG COSTFLAG(SETQ. ENDFLAG NIL) ) 


(SETQ Q11FLAG(SETQ Q12FLAG(SETQ Q13FLAG(SETQ NFLAG(SETQ AFLAG()))))) 
 (SETQ M1I9C(SETQ M18C(SETQ M17C(SETQ M16C(SETQ H1ISC(SETQ WM1IHC- | 

(SETQ H13C(SETQ M12C(SETQ H11C(SETQ M10C(SETQ 

(SETQ M8CCSETO M7CCSETQ MOC(SETQ M5C(SETQ MAC» 


HOC 


(SETQ Hed pac M2C(SETQ M1C 9)))))))))II))II))) 


(SETQ /.(QUOTE /.)) 
CSETO:../; (QUOTE 7 ,.)) 
(SETQ 2(QUOTE ?)) 
(SE LO -* (QUOTE. *)) 
(SETQ "(QUOTE ')) 


PL 


(SETQ : (QUOTE ;) 
(SETQ £(QUOTE %) 
(SETQ #( QUOTE #)) 
(SETQ SP(QUOTE / 
(SETQ ;(QUOTE ;) 
/ 
\ 
) 


= 


(SETQ /"( QUOTE 
(SETQ \( QUOTE 
(SETQ =(QUOTE = 
(SETQ LF T) 
‘SETQ NODES(LIST(SETQ ROOT(LIST 
NIL NIL NIL NIL 1 NIL NIL NIL NIL)))) 
(SETQ ARCS(SETQ PROBLEM WNIL)) 
(QUOTE SETQS) )) 
(DE STAT()(PROGCAFLAG NFLAG) 
(PRINT(QUOTE ARCS)) 
(MAPCAR( FUNCTION ioexevaiioss 
(PRINT(QUOTE NODES)) © 
(MAPCAR(FUNCTION NSTAT)NODES) 
(RETURN(QUOTE DONE)) )) 
(DE NSTAT(N) CPROG() 
(COND(NFLAG( RETURN) ) 
( (EQUAL (GETINPUT) (QUOTE (SKIP) )) (RETURN (SETO NFLAG T)))) 
(PR LF(LIST( QUOTE NAMB=) (NAME N))) | 
(PR LF(LIST(QUOTE BPF=)(BP N))) ? | 
(PR LFE(LIST( QUOTE PREDECESSOR=) (COND( (PRED N)C(NAME( PRED N))) 
(T NIL)) )) | | 
(PR LF(LIST( QUOTE SUCCESSORS= ) (COND ( (SUCC N) (MAPCAR 
(FUNCTION NAME;)(SUCC N)))(T WIL)))) 
(PR LF(LIST( QUOTE TYPE= ) (SELECTQ( TYPE N) 
(O( QUOTE TERM saiiceoee 


SLL 


121. 
lees 
Mess 
124. 


(1( QUOTE DECISION)) 
(2( QUOTE EVENT) ) 
(3(QUOTE EXPERIMENT) ) 
(QUOTE UNKNOWN)))) | | 
(PR LF(LIST( QUOTE VALUE=) (VALUE N))) 
(PR LF(LIST( QUOTE SENSITIVITY=)(SEN N))) 
(PR LF(LIST(QUOTE EXPANDEDs)(E&XP W))) 
(PR LF(LIST( QUOTE FLAG=)(FLAG N))) )) 
(DE ASTAT(A)(PROG() 
(COND(AFLAG(RETURN ) ) 
((CEQUAL(GETINPUT) (QUOTE(SKIP)))(CRETURN(SETQ AFLAG T)))) 
(PR LF(LIST( QUOTE NAME=) (NAME A))) 
(PR LF(LIST( QUOTE OBSERVATIONS=)(OBN A))) 
(PR LE(LIST(QUOTE PREDECESSOR=)(NAME(PRED A)))) 
(PR LF(LIST(QUOTE SUCCESSOR=)(NAME(SUCC A)))) 
(PR LF(LIST (QUOTE PROBABILITY=)(PROB A))) 
(PR LE(LIST(QUOTE COST=)(COST A))) 
(PR LF(LIST( QUOTE FLAG=)(FLAG A))) )) 
(DE DUMP ()(PROGN — -_ | Oo 
(PR LF (LIST(QUUTE CONCERNING) PROBLEM) ) 
(PR LFE(LIST(QUOTE(YOUR BEST DECISION IS TO)) 
(FINDANAME(MAXNODEC(SUCCNODES ROOT)1)) )) 
(HAPCAR(FUNCTION PLAN) (SUCCNODES ROOT)) )) 
(DE MAXWNODE(L C)(PROG(N) 
(COND( (NULL L)(CRETURN))) 
(SETQ N(CAR L)) | 7 
L1 (COND((LESSP(PICKV N C)(PICKV(CAR L)C))(SETQ NC(CAR L)))) 
(COND((S#TQ L(CDR L))(GO L1))) 
(RETURN N) )) 
(bE PICKV(N C)(COND 


OL 


 ((ATOM(VALUEO N))(CVALUEO N)) 
(T(CAR(NTH(VALUE WN)C))) )) 
(DE MINIL(X Y)(CONDCY(MIN X YT x) >) 
(DE TIPS(N)(PROG(TPS) 
| (TIPS*® W) | 
(RETURN TPS) )) 
(DE TIPS*(N) (COND 
((HULL(SUCC W))(SETQ TPS(CONS N TPS))) 
(T(MAPCAR( FUNCTION TIPS*)(SUCCNODES WN))))) 
(DE MEMBLISTV(L1 L2) (COND 
((WOTC(AND L1 L2))HIL) 
((AND(CATOM(CAR L1)) 
(HEMB(CAR L1)L2))T 
((ANDCNOTC(ATOM(CAR L1) 
(MEMBLIST&(CAR L1) 
(T(MEMBLISTV(CDR L1)L2 
(Dé MEMBLIST&(L1 L2) (COND 
((NULL L2)NIL) 
CONULG: L407) | 
((AND(ATOM(CAR L1)) 
(MEMB(CAR L1)L2))(MEMBLIST&(CDR L1)L2)) 
((AND(NOT(ATOM(CAR L1))) 
(MEMBLISTV(CAR L1)L2)) (HEMBLIST&(CDR L1)L2)) 
(T WIL))) 
(DE RESHUFFLE(L)(MAPCAR( FUNCTION EVAL) 
(MAPCAR(FUNCTION CONS)(CONSTANT(QUOTE PLUS))(REFORM | 
(MAPCAR( FUNCTION (LAMBDA(L) (MAPCAR(FUNCTION(LAMBDA(X Y)(TI 
(CONSTANT(CAR L))(CDR L)) })L) )))) 


ee | 


(DE BLEND(F V P C)(EVAL(CONS FCMAPCAR(FUNCTION(LAMBDA(V P C) 


(TIMES( DIFFERENCE V C)P)))V P C)))) 


ae rie | 


MES 


K Y))) 


LLL 


(DE ALLATONS(L)(EVAL(CONS( QUOTE AND) (MAPCAR(FUNCTION ATOM)L)))) 
(DE PLAN(N)(PROG(S SS) 
(SETQ S(CAR(SUCCNODES N))) 
(SETQ SS(CAR(SUCC N))) 
(COND( (NULL N)CRETURN)) 
((AND(TYPE1 N)CATOM( VALUE N)))(CPR LPCLIST (QUOTE IF) /" # 
(NAME(PRED N)) # /" (COND((TYPE1( PRED N)) (QUOTE OCCURS/,)) 
(T(QUOTE(IS CHOSEN/,)))) (NAME (PRED 
(MAXNODE(SUCCNODES N)1))) ))) 
((AND(TYPE3 N)(TYPE1 S)) 
(COND((GREATERP(CAR(VALUE S))(BLEND(FUNCTION PLUS) 
(CDR( VALUE S))(PROB SS)(COWSTANT 0O))) 
(PR LF(LIST(QUOTE(DO NOT)) (NAME SS) 
(QUOTE BUT) (NAME(PRED(MAXNODE(SUCCNODES S)1)))))) 


(T(PROG(C) 
(PR LF(LIST( NAME SS)(QUOTE AND/././.))) 
CSEPOCG 1) 

&1 (PR LE(LIST SP SP SP SP SP(QUOTE If) /" # 
(CAR(NTH(OBN SS)C)) # /" (QUOTE(IS UBSERVED/, )) 
(NAME(PRED(MAXNODE(SUCCNODES S)(PLUS C 1)))) )) 

(COND( (GREATERP(SETQ C(PLUS C 1))(LENGTHC(OBN SS))) 


(RETURN) )) 
(GO L1) ))))) 
penne PLAN) (SUCCNODES N)) )) 
(DE TRANSPOSE(L) (PROG(X) 
(SETQ X L) 
L1 (COND((NOTCATOHM(CAR X)))(GO L2)) 
((SETQ X(CDR X))(GO L1)) 
(TCRETURN L))) 
L2 (SETQ X(CAR X)) 


Sit 


(SETQ LCREMOVE X L)) 

(RETURN(MAPCAR(FUNCTION CONS) X( CONSTANT EN )) 
(DE PR(LF L)(PROG() 

(COND( (NULL L) (RETURN) )) 

(SETQ LCFLAT L)) 

L1 (COND((NOT(MEMB NIL L))(GO L2))) 

(SETQ LCREMOVE NIL L)) 

(GO 11) 


L2 (COND((NULL L)(RETURN(COND(LF(TERPRI))(T)))) 
((EQ(CAR L)#)C(CPROGN(SETQ L(CDR L))(GO L3))) 
((BQ(CAR L)\)(PROGN(SETQ L(CDR L))(TERPRI)))) 


(PRINC SP) 
L3 (PRINC(CAR L)). 
(SETQ L(CDR L)) 
(GO L2) )) 
(DE OUTCV C L)CPROG() 
(SET V(COND((EQ( EVAL V)C)1)(T(PLUS(EVAL V)1)))) 
(PR LF(CAR(NTH LCEVAL V)))) )) 
(DE M1(X)C(OUT( QUOTE M1C)5(LIST 
(LIST(QUOTE(SUPPOSE THAT YOU HAD CHOSEN TO))X # /.) 
(LIST(QUOTE(ASSUMING THAT))/" # X # /"(QUOTE(WAS PICKED/,)) ) 
(LIST(QUOTE(SUPPOSE THAT))/" # X # /"(QUOTE(IS YOUR CHOICE/ -))) 
(LIST(QUOTE(WHAT IF YOU CHOOSE TO)) X # 2?) 
(LIST(QUOTE(LET US SAY THAT YOU TOOK))/" # X # /" # 7,) 
))) 
(DE M2(X)(OUTC QUOTE M2C)4(LIST 
(LIST(QUOTE(SUPPOSE THAT))/" # X # /" (QUOTE HAPPENED/.) ) 
(LIST(QUOTE(WHAT IF))/" # X # /"( QUOTE OCCURS?) ) 
(LIST(QUOTE( ASSUME THAT))/" # X # /"(QUOTE(HAS HAPPENED/,)) ) 
(LIST(QUOTE(LETS SAY IT WAS))/" # X # /"(QUOTE( THAT ACTUALLY 


6LL 


O41, 
E42, 
243, 
244, 
245. 
Oub. 
OUT. 
24S. 
CuY., 
2a, 
oan 


ot. 


COC e 
aa ie a 
254, 
255. 
2502 
a a er 
250 
259. 
OU ss 
201. 
262. 
2v03. 
264, 
~20Os 
2bo. 
2o7. 
208. 
269. 
efu. 


TOOK PLACE/.))) 
))) 
(DE M3()(OUTC QUOTE M3C)4(LIST 
(QUOTE(IS THERE A DECISION TO BE MADE AT THIS POINT?)) 
(QUOTE( DO YOU HAVE A CHOICE OF ALTERNATIVES?) ) 
(QUOTE(ARE THERE SOME OPTIONS AVAILABLE TO YOU?7)) 
(QUOTE(WOULD THERE BE OPPORTUNITIES OPEN TO YOU NOW?)) 


(Di M4()COUTC( QUOTE M4C)4(LIST. 
(QUOTE(CAN YOU THINK OF THINGS THAT HAY HAPPEN AS A 
RESULT?) ) 
(QUOTE(ARE EVENTS ABOUT TO HAPPEN OVER WHICH YOU HAVE NO COWwTRUL?)) 
(QUOTE(ARE THERE SOM® EVENTS THAT MAY HAPPEN?) ) 
(QUOTE( COULD UNCONTROLLABLE OUTCOMES OCCUR?) ) 


(DE #5()(OUT(QUOTE M5C)5(LIST 
(QUOTE(DO YOU WISH TO STOP EXPLORING FURTHER IN THIS DIRECTIONW?)) 
(QUOTE(HAS THERE BEEN ENOUGH DETAIL EXPRESSED SO FAR?)) 
(QUOTE(SHUULD WE EXPLORE SOME OTHER POSSIBILITIES IN ANOTHER AREA?) ) 
(QUOTE( PERHAPS We SHOULD TALK ABOUT SOMETHING ELSt7?)) 
(QUOTE(I ASSUME THAT We CAN LEAVE THIS SITUATION?) ) 
pe. 
(De M6()(OUT( QUOTE M6C)6(LIST 
(QUOTE( PLEASE LIST THE ALTERNATIVES THAT YOU HAVE/, ONE AT A TIME/.)) 
(GUOTE(STATE THE CHOICES THAT YOU HAVE/.)) | 
(QUOTE(LIST THE OPTIONS THAT ARE AVAILABLE/.)) 
(QUOTE(PLEASE LIST THE DECISIONS THAT YOU COULD MAKE/.)) 
(QUOTE(TELL ME WHAT IS AVAILABLE/.)) 
(QUOTE(ENTER THE CHOICES THAT YOU COULD TAKE/.)) 
(QUOTE(WHAT OPPORTUNITIES ARE THERE?) ) 


(Odcl: 


e7l. 


oT h 
e7T5. 


Ds | oo 
(DE M70) (OUT( QUOTE H7C)5(LIST 


(QUOTE(PLEASE LIST THE OUTCOMES/.)) 

(QUOTE( EXACTLY WHAT EVENTS COULD OCCUR?)) 

(QUOTE(STATE THOSE EVENTS THAT MAY HAPPEN/.)) 
(QUOTE(LIST THE POSSIBLE OCCURANCES THAT YOU FORESEE/.)) 
(QUOTE(WHAT DO YOU SUPPOSE COULD HAPPEN?) ) 


(DE M8()C(OUTCQUOTE M8C)5(LIST 
(QUOTE(ARE THE ALTERNATIVES MUTUALLY EXCLUSIVE?) ) 


(QUOTE(DOES THE CHOICE OF ONE ALTERNATIVE EXCLUDE THE OTHERS?)) 


(QUOTE(IS ONLY ONE CHOICE POSSIBLE?)) _ 
(QUOTE(L ASSUME THAT ONLY ONE OF THE DECISIONS CAN 
BE MADE/, CORRECT?) } , 
(QUOTE(IS IT TRUB THAT CHOOSING ONE OF Sih ALTERNATIVES 
EXCLUDES THE OTHERS FROM BEING CHOSEN?) ) 
))) 


(DE 49 () COUT (QUOTE MOC)4(LIST 


(QUOTE(ARE THESE EVENTS MUTUALLY EXCLUSIVE? )) | 
(QUOTE( DOES THE OCCURANCE OF ONE EVENT #XCLUDE THE OTHERS 
\ FROM HAPPENING?) ) 


~(QUOTE(CAN JUST ONE OUTCOME HAPPEN AT A TIME?)) 


(QUOTE(I ASSUME THAT ONLY OWE CAN OCCUR/, CORRECT?)) 


)))- 
(DE el10(X)COUTC QUOTE M10C)7(LIST 


(LIST(QUOTE(LETS CONSIDER))/" # X # /*(QUOTE(FOR A MOMENTS. ))) 


(LIST(QUOTE(NOW CONSIDER))/" # X # /" # 7. 
(LIST(QUOTE(LETS LOOK AT))/" # X # 7" 7.) 
(LIST(QUOTE(WHAT ABOUT))/" # X # /" # ?) 


(LIST(QUOTE(LETS TALK ABOUT))/" # X # /"(QUOTE(FOR AWHILE/.))) 


(LIST(QUOTE(NOW CONSIDER))/" # X # /" # 7.) 
(LISTCQUOTE(NOW CONSIDER))/" # X # /" # 7.) 
))) 
(DE M110) (OUT( QUOTE M11C)5(LIST 
(QUOTE(WHAT VALUE WOULD YOU GIVE TO THIS SITUATION?)) 
(QUOTE(TRY TO PLACE A WUMERIC VALUE ON THIS SITUATIOH/.)) 
(QUOTE(HOW WOULD EVALUATE THIS SITUATION?)) 
(QUOTE(ESTIMATE THE VALUE IF YOU WERE IN THIS po OSITION/. )) 
(QUOTE(WHAT IS THE SITUATIGN WORTH TO YOU?)) 


(bE H12()(OUTC( QUOTE H12C)5(LIST 

(QGUOTEC(CTRY TO ESTIMATE THE PROBABILITY THAT THIS ®VENT WILL HAPPENS. )) 
(QUOTE( WHAT ARE THE CHANCES OF THIS OUTCOME HAPPENING?) ) 

(QUOTEC(WHAT ARE THE CHANCES THAT THIS EVENT WILL OCCUkR?)) 

(QUOTE( HOW MUCH OF A CHANCE DOES IT HAVE?)) 

(QUOTE( WHAT PROBABILITY WOULD YOU PLACE OW IT?)) 


(DE H13()COUTC QUOTE H13C)5(LIST  . 

(QUOTE (WHAT IMMEDIATE COST IS EXPECTED?) )- 

(QUOTE(WHAT WOULD BE THE IMMEDIATE COST/, ASSUMING THIS SITUATION?) ) 
(QUOTE(HOW MUCH EXPENSE IS ANTICIPATED? )) 

(QUOTE(IF THERE IS AN ASSOCIATED COST/, WHAT IS IT?)) 
(QUOTE(STATE THE IMMBDIATE COST/, IF THERE IS OWE/.)) 


))) 

(DE QYN(L) (COND 
((MEMBLISTY(QUOTE(Y YES OK RIGHT YEA))L)T) 
((MEMBLISTV(QUOTE(N NO NONE NOTHING DONE O))L)NIL) 
(T L))) | 

(DE O39(N)CPROGCANS) © 
LICCOND(C CTYPE N)(CH6)) CT CHO) )) 


col 


 (CURUDC(AUTCATOH(SETO AUSCOYNCGBTINPUT)))))CGO L1)). 
((NOT ANS) CPROGN | 
(PR LE(QUOTE(TRY TO LIST MUTUALLY EXCLUSIVE ITEMS/.))) 
(RETURN T) ))) 
(RETURN) )) 
(DE Q345(S)(PROG(ANS) | 
(COND((GREATERP S 2)(SETQ S(DIFFERENCE S 2)))) 
L1 (COND((EQ S 1)(H4)) 
((EQ S 2)(43)) 
(TC ERROR-Q345-1))). 
(COND((NOTCATOM(SETQ ANS(QYN(GETINPUT)))))(GO L1)) 
(ANS(CPROGN(SETQ ENDFLAG NIL) 
(RETURN(SELECTQ S(1 2)(2 1) CERROR-9345-2)))))) 
L2 (COND((EQ S 1)(43)) 
((EQ S 2)(M4)) 
(T( ERROR-Q345-3))) 
(CONDC(NOTCATOM(SETQ ANS(QYN(GETINPUT)))))(GO L2)) 
(AWS (PROGN(SETQ ENDFLAG NIL)C(RETURN S)))) 
L3 (M5) ~ 
(COND ( (NOT (ATOM(SETQ ANS(QYN(GETINPUT)))))(GO L3)) 
((NOT ANS)(GO L4))) 
(COND( (NOT ENDFLAG)(SETQ ENDFLAG T)) | 
(TCPROGN(PR LE (QUOTE(SHALL WE TERMINATE THE INTERVIEW?) )) 
(SETQ ENDFLAG(COND( (QYN(GETINPUT)) (QUOTE STOP) ) 
(T WIL)))))) 
(RETURN O) 


L4Y (PR LF(QUOTE(TRY TO PREDICT WHETHER YOU HAVE CONTROL OVER FUTURS 


\ EVENTS OR WHETHER THEY WILL OCCUR BY THEMSELVES/. 
\ ENTER O 1 OR 2: | 
\ 0 Ir YOU WISH TO STOP/. 


ail 


\ 1 IF YOU HAVE A DECISION TO MAKES. 
\ 2 IF UNCONTROLLABLE EVENTS WILL OCCUR/.))) 
(SETQ ANS(CAR(GETINPUT))) 
(COND((ORCEQ ANS 0)(CEQ ANS 1)(EQ ANS 2))(RETURN ANS))) 
(PR LF(QUOTE(LETS MOVE ON TO ANOTHER AREA/.))) 
(RETURN O) )) 
(bE Q67()CPROG(C 3S) 
(SETQ C 1) 
L1 (PR WIL C) , 
(COND( (NULL(QYN(CAR(SETQ S(CONS(GETINPUT)S))))) 
CRETURN(REVERSE(CDR S))) )) 
(SETQ C(PLUS 1 C)) 
(GU L1) )) 
(DE Q11() (CPRUG(ANS) 
(COND( (NULL Q11FLAG)(GO L3))) 
L1 (M11) 
L2 (SETQ ANS(FLRSTNUM(GETIWPUT))) 
(CUND( (ATOM ANS)(RETURN ANS))(T(GO L1))) 
L3 (PR LE(QUOTE(ASSUMING. THAT THE CURRENT SITUATION WERE TRUE/, \ 
~ TRY TO RATE YOUR ESTIMATION OF HOW GOOD IT IS/, \ 
INDEPENDENT Of OTHER POSSIBILITIES/. YOU viAY USE \ 
MONEY AS A SCALE OR OWE OF YOUR OWN CHOOSING/. HOWEVER/, \ 
YOU MUST BE CONSISTENT AND USE THE SAMS SCALE THROUGHOUT \ 
THE ENTIRE INTERVIEW/. \ DO YOU WISr. TO USE MONEY AS A SCALE?))) 
(SETQ COSTFLAG(QYN(GETINPUT) )) 
(PR LF(QUOTE(ENTER YOUR ESTIMATE/.))) 
(SETQ Q11FLAG T) 
(GO L2) )) 
(DE Q12()(PROG(ANS) 
(1412) 


2b 


419. 
A20. 


- (RETURN (FIRSTNUM(GETINPUT) )) )) 


(DE QG12*(L)(DIFFERENCE 1.0(EVAL(CONS( QUOTE PLUS) (REMOVE NIL L))))) 


(DE Q13() CPROG(ANS) 
(COND( (NULL Q13FLAG)(GO L3))) 
L1 (M13) © 
L2 (SETQ ANS(FIRSTNUM(GETINPUT) )) 
(COND( (ATOM ANS) (RETURN ANS))(¢T(GO L1))) 
L3 (PR LF(QUOTE(ARE THERE ANY HIDDEN COSTS NOT TAKEN INTO \ 
ACCOUNT IN THE PREVIOUS VALUE ESTIMATION? ))) 
(SETQ Q13FLAG T) 
 (COND( (NOT(QYN(GETINPUT))) (RETURN 0))) 
(GO L2) )) 
(DE GETINPUT() (PROG(L) 
L1 (SETQ L(LINEREAD) ) 
(COND( (NOT (MEMBER( QUOTE LISP)L))(GO L2))) 
(PRINT( QUOTE LISP:)) | 
(PRINT(EVAL(LINEREAD) )) 
(TERPRI) 
(GO L1)- 
L2 (RETURN Ls) )) 
(DE M14()(COUTC QUOTE M14C)3(LIST 
(QUOTE(CAN YOU IMPROVE THESE PROBABILITY ASSIGNMENTS BY \ 
PERFORMING AN EXPERIMENT?) ) 
(QUOTE(ARE YOU UNHAPPY WITH THE ACCURACY OF THESE 
PROBABILITY ESTIMATES?) ) 
(QUOTE(IS IT POSSIBLE TO DO ANYTHING THAT WILL IMPROVE \ 
THES& PROBABILITY VALUES?) ) 
y)) 
(DE M15()COUT( QUOTE M15C)1(LIST 
(QUOTE(WHAT EXPERIMENT COULD BE PERFORMED?) ) 


“Seu 


))) 
(DE H16()(OUT(QUOTE M16C)1(LIST 
(QUOTE(WHEN WOULD IT BE PERFORMED?) ) 
))) 
(DE 417()COUT(QUOTE M17C)1(LIST 


(QUOTE( PLEASE LIST THE POSSIBLE OBSERVATIONS FROM THE EXPERIMENT/.)) 


))) 
(DE M16(X)(OUT( QUOTE M18C)1(LIST 


(LISTC(QUOTE(SUPPOSE THAT))/" # X # /"(QUOTE(IS ABOUT TO 


HAPPEN/././.))) 


))) | 
(DE W19(X)(OUT( QUOTE M19C)1(LIST 
(LIST(QUOTE(WHAT IS THE PROBABILITY OF))/" 
(QUOTE HAPPENING?) ) 
))) 

(DE EXPAND(N)(PROG(S CH WNEWARC NEWNODE) 
(COND((NULL( PRED N))(GO LO))) 
((SELECTQ(SETQ S(TYPE(PREDNODE. N))) 

(1( FUNCTION 1) ) 
(2(FUNCTION M2)). 
(QUOTE ERROR-EXPAND) ) 
 (FINDANAME W)) 
~(PUTTYPE N(Q345 S)) 

LO (COND((TYPEO N)(GO L4)) 
((TYPE1 N)(CH6)) 
(T(M7)) ) 

(SETQ 3S(Q67)) 

L1 (COND((Q89°-N)(GO LO))) 

(SETQ M10C 1) 

Le (SETQ ARCS(CONS(SETQ NEWARC(LIST 


#oxX # /" 


921 


451, | (CAR S)NIL N NIL NIL NIL NIL ))ARCS)) 


W52, (ADDSUCC W NEWARC) 
W533. (M10(CAR S)) 
HOY, (SETQO NODES(CONS(SETQ NEWNODE(LIST 
H55. WIL NIL NEWARC NIL WIL(Q11)NIL NIL NIL))NODES)) 
4S6, (PUTSUCC NEWARC NEWNODE) 
i L3 (COND((TYPE2 N)(PUTPROB NEWARC(COND( (CDR S)(Q12)) 
458. (T(Q12*(PROBS(SUCC WN))))) ))) 
459. (COND(COSTFLAG(PUTCOST WEWARC(Q13)))) 
H650. (COND((SETQ S(CDR S))(GO L2))) 
Hol. (COND((TYPE2 N) (EXPERIMENT N))) 
HO2. L4 (RETURN(PUTEXP W T)) )) 
463. (DE START()(PROGN © 
Hoy, (SETQS) 
Hos, (PR NILC(QUOTE(WHAT/'S YOUR PROBLEM? ) ) ) 
466... ~ (SETQ PROBLEM(GETINPUT) ) : 
HOT. (EXPAND ROOT) 
4HOo8. (MAPCAR(FUNCTION EXPAND) (SUCCNODES ROOT) ) 
Hog. (START* ROOT) 
HE Oh: (DUMP) 
471. (QUOTE "PROGRAM FINISHED") )) 
472. (DE START*®(N)CPROG() 
NTs. (COND ( (NULL( EXP OER ERNE NN) )) 
u74, L1 (ROLLBACK) 
475. (SENSITIVITY) 
ame (EXPAND(COND( (WODE-SELECT WN))(TCRETURN)))). 
U77. (COND(CEQ( QUOTE STOP) ENDFLAG) (RETURN) ) ) 
N73. (GO L1) )) 
u79. (DE ROLLBACK()(PROG() 


430. (MAPCAR( FUNCTION (LAMBDA(N)(PUTFLAG WN 


rAS 


(OR(NULL(TYPE N))(TYPEO H)))))NODES) 
L1 (COND((EVAL(CONS( QUOTE OR) (MAPCAR(FUNCTION ROLLCALC) 
NODES)))(GO L1))) )) 
(DE WODRE-SELECT(SS)(PROG(N S) 
(SETQ S (TIPS SS)) | 
L1 (COND( (NULL S)(COND(N(RETURN N))C(T(GO L2)))) 
(CANDCNULL(SEN(CAR S)))(NOTCTYPEO(CAR S)))) 
(RETURN(CAR S)) ) 
((CORC(CTYPEO(CAR S))(NULL(SEN(CAR S)))CZEROP(S&EN(CAR S))))WNIL) 
((OR(NULL WN) (LESSP(SEN(CAR 8))(SEW N))) 
(SETO W(CAR S)) )) 
(SETQ S(CDR S)) | 
(GO L1) 
Le (SETO SCTLPS 8S)) 


L3 (COND((AND(ZEROP(SEN(CAR S)))(NOT(TYPEO(CAR S))))CRETURN(CARK S))) 
((SETQ S(CDR S))(GO L3))) 


(RETURN) )) 

(DE SENSITIVITY()(PROG(H V) _ | 
(MAPCAR(FUNCTION(LAMBDA(X) (PUTSEN X NIL)))NODES) 
(SETQ MCEVAL(CONS( QUOTE (MAX) (SETQ VCVALUES(SUCCNODES ROOT)))))) 
(HAPCAR(C FUNCTION SENCALC) 

(SUCCNODBS ROOT) 
(CONSTANT 1.0) 
(MAPCAR( FUNCTION (LAMBDA(X) (DIFFERENCE M %)))V)) )) 

(DE SENCALC(N P DR)C(PROG(SN) 

(PUTSEN NCSETO SN( QUOTIENT DR P))) 
(CUnD((NULL(SUCC W)) (RETURN) )) 
(MAPCAR(FUNCTION SENCALC) 
(SUCCNODES WN) 
(COND((TYPE1 N)CCONSTANT 1.0)) 


82k 


((TYPE2 N)(MAPCAR(FUNCTION(LAMBDA(X Y)(TIMES X Y))) 
-(PROB1S(SUCC W)) 
(CONSTANT P)))) 
(COND ( (TYPE? N) (CONSTANT DR)) 

((TYPET N) (HAPCAR( FUNCTION (LAMBDA(X) 
(DIFFERENCE(PLUS SN(VALUE WN))X))) 
(VALUES(SUCCNODES N)) ))) ))) 

(Dig CONSTANT(X) (PROG( XX) (RETURNC(RPLACD(SETQ XX(LIST X))XX)))) 
(DE ROLLCALC(N)(PROG(V PC F) : 
(COND((EVAL(CONS(QUOTE OR)(CONS(FLAG N) 
(MAPCAR(FUNCTION NOT) (FLAGS(SUCCNODES Nw) ))))) (RETURN) )) 
(SETQ V(VALUES(SUCCNODES WN))) 
(SETO P(PROB1S(SUCC N))) 
(SETQ C(COSTS(SUCC N))) - 
(SETQ F(COND((TYPE1 WN) (QUOTE MAX) ) (E(QUOTE PLUS)))) 
(PUTVALUE N(COND | 
((TYPE3 N)(MAX(CAAR V)(BLEND( QUOTE PLUS) (CDAR V) 
(CAR P)(CONSTANT(CAR C))))) 
((ANDCATOM(CAR P))C(ALLATOMS V))(BLEND F V P C)) 
(T(HAPCAR(FUNCTION BLEND) (CONSTANT F) 
(COND((TYPE1 N)( TRANSPOSE V))(TC(CONSTANT V))) 
(COND((TYPE1 N)C(CONSTANT P))(TCREFORM P))) 
(CONSTANT C) )) )) 
(RETURN(PUTFLAG N T)) )) 
(DE REFORM(L) (HAPCAR (FUNCTION (LAMBDA (C) 
(MAPCAR(FUNCTION CAR) 
(MAPCAR( FUNCTION WNTH)L(CONSTANT C)) ))) 
(REVERSE (MAPLIST(FUNCTION LENGTH) (CAR L))) )) 
(DE EXPERIMENT(N)(PROG(NN wW EXPARC EXPNODE C ER E) 
LO (M14) | 


(6cL 


(COND ( (NOT CATOM(SETQ NN COYN(GETINPUT))))) (GO 6) 
(CNOT WN) CRETURN) )) 


(115) | 
(SETQ NW(GETINPUT)) 
(M16) 


(SETQ W(FPINDEXPNODE(GETINPUT)N) ) 

(SETQ EXPARC(LIST NN NIL WIL(SUCC W)NIL WIL NIL)) 

(SETQ EXPNODE(LIST WIL NIL WC(LIST EXPARC)3 NIL NIL WIL wIL)) 

(PUTPRED(SUCC EXPARC)EXPARC) : 

(PUTPRED EXPARC BXPNODE) 

(PUTSUCC W BXPNODE) 

(SETQ NODES(CONS EXPNODE NODES) ) 

(SETQ ARCS(CONS EXPARC ARCS)) 

(M17) . 

(SETQ C(SUCC N)) 

L3 (SETQ EE(PUTOBN EXPARC(Q67))) 

(COND((Q69 N)(GO L3))) 

L1 (SETOQ E BE) | 

(i418 (NAME CCAR C))). 

b2 (M19(CAR E£)) | 

(ADDPROB(CAR C)(FIRSTNUM(GRTINPUT))). 

(COND((SETQ ECCDR £))(COND( (CDR E)(GO L2)) 
(TCADDPROB(CAR C)(DIFFERENCE 1.0(EVAL(CONS( QUOTE PLUS) 

(CDR(PROB(CAR C))))))))))) 

(COND((SETG C (CDR C))(GO L1))) | 

(PUTPROB EXPARC(RESHUFFLE(PROBS(SUCC W)))) 

(MAPCAR(FUNCTION PUTPROB)(SUCC WN) 
(MAPCAR(FUNCTION CONS) (MAPCAR(FUNCTION CAR) 
(PROBS(SUCC W)))(MAPCAR(FUNCTION BAYES) 
“(PROBS(SUCC W))C(CONSTANT( PROB EXPARC)) )))- 


OEL- 


(RETURN) )) 
(DE FINDEXPNODE(FIND N)(PROG(ARC SCORE HARC HSCORE WORDS) 


(SETQ HARC(SETQ ARC(PRED W))) 

(SETQ HSCORE 0) | 

L1 (SETQ WORDS(NAME ARC) ) 

(SETQ SCORE 0) 

L2 (COND((MEMBER(CAR WORDS)FIND)(SETQ SCORE( PLUS 1 SCORK)))) 

(COND((SETQ WORDS(CDR WORDS))(GO L2)) 
((LESSP SCORE HSCORE)(GO L3))) 

(SETQ HSCORE SCORE) 

(SETQ HARC ARC) 

L3 (COND((SETQ ARC(PRED( PRED ARC)))(GO L1))) 

(RETURN(COND( (ZEROP HSCORE)NIL)(T HARC))) )) 


(DE BAYES(CA B)(MAPCAR(FUNCTION(LAMBDA(X Y 2) 


(QUOTIENT(TIMES X Y)Z)))(CDR A)(CONSTANT(CAR A))B)) 


